Centralized management center for managing storage services

ABSTRACT

Methods and systems for providing storage services in a networked environment are provided. A management device interfaces with a plurality of management layers that communicates with a plurality of application plugins executed by a plurality of computing devices. Each application plugin is associated with an application for providing storage services for stored objects managed by a storage system. A same request and response format is used by the management device to obtain information from the plurality of management layers regarding storage space used by the plurality of applications for storing the stored objects and the management device maintains storage space information as a storage resource object for virtual storage resources and physical storage resources used by the plurality of applications for storing the stored objects.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application claims priority under 35 USC § 119 (e) to USProvisional Patent Application Entitled “Centralized Management CenterFor Managing Services” Ser. No. 62/098,601 filed on Dec. 31, 2014, thedisclosure of which is incorporated herein by reference in its entirety.

COPYRIGHT NOTICE

A portion of the disclosure herein contains material to which a claimfor copyrights is made. The copyright owner, the assignee of this patentapplication, does not have any objection to the facsimile reproductionof any patent document as it appears in the USPTO patent files orrecords, but reserves all other copyrights, whatsoever.

TECHNICAL FIELD

The present disclosure relates to storage systems and more particularlyto, centralized management of storage services for a plurality ofapplications in different operating environments.

BACKGROUND

Various forms of storage systems are used today. These forms includedirect attached storage (DAS) network attached storage (NAS) systems,storage area networks (SANs), and others. Network storage systems arecommonly used for a variety of purposes, such as providing multipleusers with access to shared data, backing up data and others.

A storage system typically includes at least one computing systemexecuting a storage operating system for storing and retrieving data onbehalf of one or more client computing systems (“clients”). The storageoperating system stores and manages shared data containers in a set ofmass storage devices.

Storage systems are being used extensively by different applications,for example, electronic mail (email) servers, database applications,virtual machines executed within virtual machine environments (forexample, a hypervisor operating environment) and others to store data,protect the data using backups and cloning. Different applicationsexecuted at various computing system types have varying requirements forstoring information, protecting it by backups and then using restoreprocedures to restore such backups. The different applications may usedifferent parameters for role based access control to access storageservices, storage space or stored data containers (for example, files,directories, structured or unstructured data). Managing storedapplication objects and providing storage services, for example,backups, restore, cloning and other services in such an environment is achallenge. Continuous efforts are being made to better manage storageservices.

SUMMARY

In one aspect, a machine implemented method is provided. The methodincludes interfacing by a management device with a plurality ofmanagement layers that communicate with a plurality of applicationplugins executed by a plurality of computing devices, where eachapplication plugin is associated with an application for providingstorage services for stored objects managed by a storage system for theplurality of applications; and using a same request and response formatby the management device to obtain information from the plurality ofmanagement layers regarding storage space used by the plurality ofapplications for storing the stored objects.

The method further includes maintaining storage space information as astorage resource object by the management device for virtual storageresources and physical storage resources used by the plurality ofapplications for storing the stored objects; and using a base backuppolicy for different stored object types and application specific policyfor each application associated with different stored object types forproviding storage services by the management device for the differentstored objects.

In another aspect, a non-transitory, machine readable storage mediumhaving stored thereon instructions for performing a method is provided.The machine executable code which when executed by at least one machine,causes the machine to: interface by a management device with a pluralityof management layers that communicates with a plurality of applicationplugins executed by a plurality of computing devices, where eachapplication plugin is associated with an application for providingstorage services for stored objects managed by a storage system for theplurality of applications; and use a same request and response format bythe management device to obtain information from the plurality ofmanagement layers regarding storage space used by the plurality ofapplications for storing the stored objects.

The machine executable code further causes the machine to maintainstorage space information as a storage resource object by the managementdevice for virtual storage resources and physical storage resources usedby the plurality of applications for storing the stored objects; and usea base backup policy for different stored object types and applicationspecific policy for each application associated with different storedobject types for providing storage services by the management device forthe different stored objects.

In yet another aspect, a system having a memory with machine readablemedium comprising machine executable code having stored thereoninstructions is provided. A processor module of a management devicecoupled to the memory, executes the machine executable code to:interface with a plurality of management layers that communicate with aplurality of application plugins executed by a plurality of computingdevices, where each application plugin is associated with an applicationfor providing storage services for stored objects managed by a storagesystem for the plurality of applications; and use a same request andresponse format by the management device to obtain information from theplurality of management layers regarding storage space used by theplurality of applications for storing the stored objects.

The machine executable code further maintains storage space informationas a storage resource object by the management device for virtualstorage resources and physical storage resources used by the pluralityof applications for storing the stored objects; and uses a base backuppolicy for different stored object types and application specific policybased for each application associated with different stored object typesfor providing storage services by the management device for thedifferent stored objects.

This brief summary has been provided so that the nature of thisdisclosure may be understood quickly. A more complete understanding ofthe disclosure can be obtained by reference to the following detaileddescription of the various aspects thereof in connection with theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features and other features will now be described withreference to the drawings of the various aspects of the presentdisclosure. In the drawings, the same components have the same referencenumerals. The illustrated aspects are intended to illustrate, but not tolimit the present disclosure. The drawings include the followingFigures:

FIG. 1A shows an example of an operating environment for the variousaspects disclosed herein;

FIG. 1B shows an example of presenting storage space to a virtualmachine, according to one aspect;

FIG. 1C shows a block diagram of a snap manager (SM) module used by acentralized snap manager server (“SMS”), according to one aspect of thepresent disclosure;

FIG. 1D shows a block diagram of a snap manager core (SMcore) layer usedby a computing system (or host system) for interfacing with SMS andother components' of FIG. 1A, according to one aspect of the presentdisclosure;

FIG. 1E shows an example of different host systems interfacing with theSMS of the present disclosure;

FIG. 1F shows an illustration of SMS interfacing with different pluginsfor different applications, according to one aspect of the presentdisclosure;

FIG. 2A shows an example of a request and response format forcommunication between the SMS and SMcore layers, according to one aspectof the present disclosure;

FIG. 2B shows an example of an object format used by SMS to storeinformation regarding different applications, according to one aspect ofthe disclosure;

FIG. 2C shows a storage layout maintained by SMS to provide storageservices, according to one aspect of the present disclosure;

FIG. 2D shows an example of a storage resource object, according to oneaspect of the present disclosure;

FIG. 2E shows an example of a dataset object maintained by the SMS forproviding storage services, according to one aspect of the presentdisclosure;

FIG. 2F shows a process flow for creating a dataset, according to oneaspect of the present disclosure;

FIG. 3A shows an example of installing a plugin, according to one aspectof the present disclosure;

FIG. 3B shows a discovery process flow, according to one aspect of thepresent disclosure;

FIG. 3C shows a process for resource discovery involving differentapplications, plugins and the SMS, according to one aspect of thepresent disclosure;

FIG. 3D shows an example of a process for discovery resources involvinga Windows based host, according to one aspect of the present disclosure;

FIG. 3E shows an example of inventory management using SMS and othercomponents, according to one aspect of the present disclosure;

FIG. 4A shows a role based access control (RBAC) format that is used bySMS to provide storage services, according to one aspect of the presentdisclosure;

FIG. 4B shows a detailed example of the RBAC format used by the SMS,according to one aspect of the present disclosure;

FIG. 4C shows a process flow for configuring RBAC parameters by SMS,according to one aspect of the present disclosure;

FIG. 5A shows a process flow for creating a dataset and then executing abackup workflow, according to one aspect of the present disclosure;

FIG. 5B shows a process flow for taking backups in the system of FIG.1A, according to one aspect of the present disclosure;

FIG. 5C shows portions of the process of FIG. 5B in a non-virtualizedenvironment, according to one aspect of the present disclosure;

FIG. 5D shows a format for storing backup metadata by the SMS, accordingto one aspect of the present disclosure;

FIG. 6A shows a restore process flow in a non-virtualized environment bythe SMS, according to one aspect of the present disclosure;

FIG. 6B shows a restore process flow in a virtualized environment by theSMS, according to one aspect of the present disclosure;

FIG. 6C shows a restore process flow in a virtualized environment by theSMS for a partial virtual disk, according to one aspect of the presentdisclosure;

FIG. 7A shows a high level process flow for implementing clone lifecycle management, according to one aspect of the present disclosure;

FIG. 7B shows an example of a clone object format used by the SMS,according to one aspect of the present disclosure;

FIG. 7C shows a clone refresh process flow, according to one aspect ofthe present disclosure;

FIG. 7D shows a process flow for generating a clone, according to oneaspect of the present disclosure;

FIG. 7E (7E-1 to 7E-3) shows a process flow for generating a clone froma primary storage using a virtualization plugin, according to one aspectof the present disclosure;

FIG. 7F shows a process flow for deleting a clone, according to oneaspect of the present disclosure;

FIG. 8A shows an example of a GUI dashboard, according to one aspect ofthe present disclosure;

FIG. 8B shows an example of host system view, according to one aspect ofthe present disclosure;

FIG. 8C shows an example of an inventory view, according to one aspectof the present disclosure;

FIG. 8D shows an example of a dataset view provided by the SMS,according to one aspect of the present disclosure;

FIG. 8E shows an example of a GUI screenshot for adding a new dataset,according to one aspect of the present disclosure;

FIG. 8F shows an example of a GUI screenshot showing a policy object,according to one aspect of the present disclosure;

FIG. 8G shows an example of a GUI screenshot for adding a new policyobject, according to one aspect of the present disclosure;

FIG. 8H shows an example of a GUI screenshot for adding a schedule forthe new policy object of FIG. 8G, according to one aspect of the presentdisclosure;

FIG. 8I shows an example of a GUI screenshot for adding a retentionpolicy for the new policy object of FIG. 8G, according to one aspect ofthe present disclosure;

FIG. 8J shows an example of a GUI screenshot for adding a replicationpolicy for the new policy object of FIG. 8G, according to one aspect ofthe present disclosure;

FIG. 8K shows an example of a GUI screenshot for adding a script for thenew policy object of FIG. 8G, according to one aspect of the presentdisclosure;

FIG. 8L shows an example of a GUI screenshot for adding a backup typefor the new policy object of FIG. 8G, according to one aspect of thepresent disclosure;

FIG. 8M shows an example of a GUI screenshot for adding a verificationtype for the new policy object of FIG. 8G, according to one aspect ofthe present disclosure;

FIG. 8N shows an example of a GUI screenshot for adding an availabilitygroup for the new policy object of FIG. 8G, according to one aspect ofthe present disclosure;

FIG. 8O shows an example of a GUI screenshot showing a summary for thenew policy object of FIG. 8G, according to one aspect of the presentdisclosure;

FIG. 8P shows an example of a GUI screenshot showing various jobs thatare managed by the SMS, according to one aspect of the presentdisclosure;

FIG. 8Q shows an example of a GUI screenshot of a reports pane providedby the SMS, according to one aspect of the present disclosure;

FIG. 8R shows another example of a reports pane provided by the SMS,according to one aspect of the present disclosure;

FIG. 8S shows an example of an administration GUI screenshot provided bythe SMS, according to one aspect of the present disclosure;

FIG. 8T shows an example of a settings GUI screenshot provided by theSMS, according to one aspect of the present disclosure;

FIG. 9 shows an example of a storage system node, used according to oneaspect of the present disclosure;

FIG. 10 shows an example of a storage operating system, used accordingto one aspect of the present disclosure; and

FIG. 11 shows an example of a processing system, used according to oneaspect of the present disclosure.

DETAILED DESCRIPTION

As preliminary note, the terms “component”, “module”, “system,” and thelike as used herein are intended to refer to a computer-related entity,either software-executing general purpose processor, hardware, firmwareand a combination thereof. For example, a component may be, but is notlimited to being, a process running on a processor, a hardware basedprocessor, an object, an executable, a thread of execution, a program,and/or a computer.

By way of illustration, both an application running on a server and theserver can be a component. One or more components may reside within aprocess and/or thread of execution, and a component may be localized onone computer and/or distributed between two or more computers. Also,these components can execute from various computer readable media havingvarious data structures stored thereon. The components may communicatevia local and/or remote processes such as in accordance with a signalhaving one or more data packets (e.g., data from one componentinteracting with another component in a local system, distributedsystem, and/or across a network such as the Internet with other systemsvia the signal).

Computer executable components can be stored, for example, atnon-transitory, computer readable media including, but not limited to,an ASIC (application specific integrated circuit), CD (compact disc),DVD (digital video disk), ROM (read only memory), floppy disk, harddisk, EEPROM (electrically erasable programmable read only memory),memory stick or any other storage device, in accordance with the claimedsubject matter.

System 100:

FIG. 1A shows an example of a system 100, where the various adaptiveaspects disclosed herein may be implemented. In one aspect, system 100includes at least a host computing system 102 (shown as host systems102A-102N and may also be referred to as a host platform 102 or simplyas server 102) communicably coupled to a storage system 120 executing astorage operating system 124 via a connection system 109 such as a localarea network (LAN), wide area network (WAN), the Internet and others. Asdescribed herein, the term “communicably coupled” may refer to a directconnection, a network connection, or other connections to enablecommunication between computing and network devices.

System 100 also includes a centralized snap manager server (alsoreferred to as “SMS” or SM Server) 132 that executes instructions for asnap manager module (“SM module”) 134 for coordinating storage servicesrelated operations (for example, backups, restore, cloning and otheroperations) for different applications and plugins, as described belowin more detail. Although SMS 132 is shown as a stand-alone module, itmay be implemented with other applications, for example, within avirtual machine environment, as described below.

It is also noteworthy that SMS 132 is referred to as a snap managerserver as an example, and may be referred to or described usingdifferent terminology (for example, a central snap server, a centralstorage services provider and other descriptive terms). The variousaspects described herein are of course not limited by how SMS 132 iscategorized or the terminology used to describe its innovativefunctionality, described below in more detail. Furthermore, SMS 132 andthe SM module 134 may be referred to interchangeably throughout thisspecification.

Host systems 102 may execute a plurality of applications 126A-126N, forexample, an email server (Exchange server), a database application (forexample, SQL database application, Oracle database application andothers) and others. These applications may be executed in differentoperating environments, for example, a virtual machine environment(described below), Windows, Solaris, Unix and others. The applicationsmay use storage system 120 to store information at storage devices 114.

To protect information associated with each application, a plugin module(shown as application plugin 128A-128N) are provided. The term protectmeans to backup an application and/or backup associated information(including configuration information, data (files, directories,structured or unstructured data) and others (may jointly be referred toas data containers)).

Each host system also executes a snap manager core (also referred to asSMcore) layer 130A-130N (may be referred to as SMcore layer 130 orSMcore layers 130) that interfaces with SMS 132 and the variousapplication plugins for managing backups, restore, cloning and otheroperations, as described below in detail.

In one aspect, the storage system 120 has access to a set of massstorage devices 114A-114N (may be referred to as storage devices 114)within at least one storage subsystem 116. The storage devices 114 mayinclude writable storage device media such as magnetic disks, videotape, optical, DVD, magnetic tape, non-volatile memory devices forexample, self-encrypting drives, flash memory devices and any othersimilar media adapted to store structured or non-structured data. Thestorage devices 114 may be organized as one or more groups of RedundantArray of Independent (or Inexpensive) Disks (RAID). The various aspectsdisclosed are not limited to any particular storage device or storagedevice configuration.

The storage system 120 provides a set of storage volumes to the hostsystems 102 via connection system 109. The storage operating system 124can present or export data stored at storage devices 114 as a volume (orlogical unit number (LUN)). Each volume may be configured to store datafiles (or data containers or data objects), scripts, word processingdocuments, executable programs, and any other type of structured orunstructured data. From the perspective of one of the client systems,each volume can appear to be a single storage drive. However, eachvolume can represent the storage space in one storage device, anaggregate of some or all of the storage space in multiple storagedevices, a RAID group, or any other suitable set of storage space. Anaggregate is typically managed by a storage operating system 124 andidentified by a unique identifier (not shown). It is noteworthy that theterm “disk” as used herein is intended to mean any storage device/spaceand not to limit the adaptive aspects to any particular type of storagedevice, for example, hard disks.

The storage system 120 may be used to store and manage information atstorage devices 114 based on a request generated by an applicationexecuted by a host system or any other entity. The request may be basedon file-based access protocols, for example, the Common Internet FileSystem (CIFS) protocol or Network File System (NFS) protocol, over theTransmission Control Protocol/Internet Protocol (TCP/IP). Alternatively,the request may use block-based access protocols, for example, the SmallComputer Systems Interface (SCSI) protocol encapsulated over TCP (iSCSI)and SCSI encapsulated over Fibre Channel (FC).

In a typical mode of operation, one or more input/output (I/O) commands,such as an NFS or CIFS request, is sent over connection system 109 tothe storage system 120. Storage system 120 receives the request, issuesone or more I/O commands to storage devices 114 to read or write thedata on behalf of the client system, and issues an NFS or CIFS responsecontaining the requested data over the network 109 to the respectiveclient system.

Although storage system 120 is shown as a stand-alone system, i.e. anon-cluster based system, in another aspect, storage system 120 may havea distributed architecture; for example, a cluster based system that mayinclude a separate N-(“network”) blade and D-(disk) blade. Briefly, theN-blade is used to communicate with host platforms 102, while theD-blade is used to communicate with the storage devices 114. The N-bladeand D-blade may communicate with each other using an internal protocol.

Alternatively, storage system 120 may have an integrated architecture,where the network and data components are included within a singlechassis. The storage system 120 further may be coupled through aswitching fabric to other similar storage systems (not shown) which havetheir own local storage subsystems. In this way, all of the storagesubsystems can form a single storage pool, to which any client of any ofthe storage servers has access.

Storage system 120 also executes or includes a storage services module122 that coordinates storage volume backups, cloning, restore andreplication for different hosts and different applications. Althoughstorage services module 122 is shown as a single block, it may includevarious modules to taking backups, cloning restore operations,replicating backups from one location to another and so forth. Asdescribed below, backups and other operations may be performed using SMS132 and snap manager module (“SM module) 134. As an example, takingbackups may include taking snapshots, i.e. a point-in-time copy of astorage volume. The point-in-time copy captures all the information in astorage volume. The snapshot may be used to restore a storage volume atany given time.

Storage system 120 also protects snapshots by replicating snapshotsstored at a first storage system (may be referred to as primary storage)and replicating it to a secondary storage source. Differenttechnologies, including the SnapVault and SnapMirror technologies ofNetApp Inc. (without derogation of any trademark rights of NetApp Inc.)may be used to protect storage volumes.

SnapVault is primarily intended for disk-to-disk backups. SnapVaultleverages NetApp Snapshot technology to back up and restore systems at ablock level. SnapVault identifies and copies only the changed blocks ofa system to secondary storage. SnapMirror takes a mirror copy of astorage volume and stores it at a remote location/disaster recoverysite. SnapMirror can occur either at volume level or at a Qtree level.The various techniques described herein are not limited to any specificreplication protection technology.

One or more of the host systems (for example, 102A) may execute avirtual machine environment where a physical resource is time-sharedamong a plurality of independently operating processor executablevirtual machines (also referred to as VMs). Each VM may function as aself-contained platform, running its own operating system (OS) andcomputer executable, application software. The computer executableinstructions running in a VM may be collectively referred to herein as“guest software.” In addition, resources available within the VM may bereferred to herein as “guest resources.”

The guest software expects to operate as if it were running on adedicated computer rather than in a VM. That is, the guest softwareexpects to control various events and have access to hardware resourceson a physical computing system (may also be referred to as a hostplatform) which may be referred to herein as “host hardware resources”.The host hardware resource may include one or more processors, resourcesresident on the processors (e.g., control registers, caches and others),memory (instructions residing in memory, e.g., descriptor tables), andother resources (e.g., input/output devices, host attached storage,network attached storage or other like storage) that reside in aphysical machine or are coupled to the host platform.

Host platform 102A provides a processor executable virtual machineenvironment executing a plurality of VMs 112A-112N. VMs 112A-112N thatexecute a plurality of guest OS 104A-104N (may also be referred to asguest OS 104) that share hardware resources 110. As described above,hardware resources 110 may include CPU, memory, I/O devices, storage orany other hardware resource.

In one aspect, host platform 102A may also include a virtual machinemonitor (VMM) 106, for example, a processor executed hypervisor layerprovided by VMWare Inc., Hyper-V layer provided by Microsoft Corporationof Redmond, Wash. or any other layer type. VMM 106 presents and managesthe plurality of guest OS 104A-104N executed by the host platform 102A.

In one aspect, VMM 106 is executed by host platform 102A with VMs112A-112N. In another aspect, VMM 106 may be executed by an independentstand-alone computing system, often referred to as a hypervisor serveror VMM server and VMs 112A-112N are presented on another computingsystem.

It is noteworthy that various vendors provide virtualizationenvironments, for example, VMware Corporation, Microsoft Corporation andothers. The generic virtualization environment described above withrespect to FIG. 1A may be customized depending on the virtualenvironment provider.

VMM 106 may include or interface with a virtualization layer (VIL) 108that provides one or more virtualized hardware resource 110 to each OS104A-104N. VMM 106 also includes or interfaces with a hypervisor plugin(shown as application plugin 128A) and the SMcore layer 130A that aredescribed below in detail.

To manage virtual and physical resources, system 100 may include avirtual center management console (may be referred to as VCenter(provided by Microsoft Corporation) or VSphere (provided by VMWare Inc.)(without derogation of any third party trademark rights) 103. Themanagement console may execute a management application 126A (referredto as virtual storage console (VSC)) for enabling monitoring of hostconfiguration, provisioning of data stores, application cloning as wellas backup and recovery services. VSC 126A is used to manage the storageused by the various VMs. A VSC plugin 185A interfaces between the VSCand other components, as described below in detail. The term VSC pluginand hypervisor plugin are used interchangeably. As an example, the VSCplugin 185A may be executed within VMM, for example, as 128A.

Virtual Storage:

Before describing the details of the various aspects of the presentdisclosure, the following provides an example of presenting logicalstorage space to one or more VMs with respect to FIG. 1B. Storage system120 typically presents storage space at storage device 114 as a LUN toVMM 106. For example, LUN-A 138A and LUN-B 138B at storage device 114for volume “vol1” are presented to VMM 106 that hosts a plurality of VMs112A (VM1)-112B (VM2).

A file system for example, a NTFS file system (used in a Windows®operating system environment) is created (for example, by the storagesystem interface 195, FIG. 1D) on the LUNs and one or more virtual harddrive (VHD) files are also generated for each LUN. The user is presentedwith a storage drive within a virtual machine. For example, the VHD fileVM1.VHD 136A is created on LUN-A 138A and then presented as drive K:\ toVM1 112A. A user using VM1 112A uses K:\ to access storage space forreading and writing information. Similarly, VM2.VHD 136B is created onLUN-B 138B and appears as M:\ drive for VM2 112B. A user using VM2 112Buses M:\ drive to store information.

In some instances, a file system for the LUNs is not created by thestorage system interface 195 and instead the LUNs are presented directlyto the VM as a storage drive. In such a case, the file system may becreated by the guest OS. The storage drives in such an instance may bereferred to as “pass through” disks. The terms VHD and pass throughdisks as used herein for presenting a virtual storage drive to a uservia a VM are used interchangeably throughout this specification.

SMS 132:

FIG. 1C shows a block-level diagram of SMS 132 having a SM module 134,according to one aspect. The SM module 134 may be executed by astand-alone system or may interface or is integrated with anothermanagement console/application to manage and interface with multipleinstances of SMcore layers 130A-130N, as described below in detail. SMmodule 134 may also be implemented as an application within a VMenvironment.

The SM module 134 includes a graphical user interface (GUI) module 142that presents a GUI at a display device, for example, a monitor, a smartphone, tablet or any other display device type. The GUIs may be used bydifferent users to interface with SMS 132 and its components. Examplesof various GUIs are provided in FIGS. 8A-8T and described below indetail. It is noteworthy that the various aspects described herein arenot limited to any specific GUI type because a command line interface(CLI) may also be used to implement the adaptive aspects describedherein.

The SM module 134 may also include a web server 144 for enablingInternet based communication. As an example, web server 144 may beimplemented as an Internet Information Services (IIS) for a Windows®Server web server (without derogation of any third party trademarkrights). The web server 144 interfaces with a workflow engine 158 thatcoordinates and manages various tasks that are performed by thedifferent components of SM module 134 as described below in detail.

In one aspect, the workflow engine 158 coordinates dataset 164 creation,policy allocation and manage a database 176 that is described below indetail. The workflow engine 158 will also communicate with variousSMcore layers 130A-130N for host system related operations, as describedbelow.

In one aspect, the SM module 134 also includes a protection module 146,a provisioning module 148, a discovery module 150, a recovery module152, a cloning module 154, a role based access control (RBAC) module156, a storage abstraction layer (may also be referred to as “SAL”) 151,a hypervisor abstraction layer (may also be referred to as “HAL”) 153, ascheduler 160, a job manager 182, a remote installation module 166, alog manager 168, a policy data structure 170, a reports module 172 and adatabase access layer 174 that can access the database (or any otherdata structure type) 176. The database 176 stores various datastructures (or objects) in a format that allows SM module 134 to handlestorage services for different applications/host systems, as describedbelow in detail.

The log manager 168 collects logs from SMcore layers and the variousplugins. The logs can then be presented to a user via a GUI. The logsmay be for event management and audit for various SMS 132 operations.The logs may be collected for a job based on a job identifier, asdescribed below.

The protection module 146 is used to enforce a policy for a particularstorage services related job (for example, a backup operation). Theprotection module maintains a protection policy for a plurality ofobjects (or protection group) that are to be backed up and providesprotection service for backing up the protection group objects. Based onan application object, a call is made to an appropriate plugin forproviding the appropriate protection service, as described below indetail.

In one aspect, protection module 146 maintains protection group objectsfor abstract representation of a container of application objects whereprotection of application objects is defined by policy objects. Theprotection group objects map to dataset objects 164, as described belowin detail.

The provisioning module 148 allows a user to configure and provision aLUN/volume (used interchangeably) that may be used to store information.The provisioning module 148 allows a user to set a LUN size andappropriate permissions for using the LUN, for example, reading andwriting data, permission for changing a LUN size, deleting a LUN andother operations. Storage volume information is saved in a standardformat at database 176 and includes, name of the storage volume, storageconnection identifier (described below), size, a junction path, datevolume was created and an aggregate. It is noteworthy that theprovisioning module 146 may be a part of another management application,including VSC 126A.

The discovery module 150 interfaces with the SMcore layers 130A-130Nexecuted at different host systems to obtain information regarding thehost systems, storage resources used by various applications andinformation regarding data containers that are protected (i.e. backedup) and unprotected. The discovery module 150 also facilitates discoveryand management of application specific objects, for example, VMs,databases, hypervisor and others. Based on the application type, anappropriate plugin is used to discover different objects, as describedbelow in detail.

In one aspect, the discovery module 150 initiates a discovery operationwith the SMcore layers. An application programming interface (API)presented by the SMS 132 determines if a SMcore layer is installed at ahost. If the SMcore is installed, then the SMcore discovers the variousplugins at that host. If the SMcore layer is not installed, then theSMcore layer is installed by the remote installation module 166 and theappropriate plugins are installed as well.

To discover application resources, a user is authenticated by SMS 132,as described below. A user token is generated and the same token is thenused across multiple hosts to discover application plugin resources asdescribed below in detail.

The cloning module 154 is used to clone storage volumes that aremaintained by the storage system 120. The cloning module 154 is alsoused for managing the life cycle of a clone, as described below indetail. The term clone as used herein is a duplicate copy of a snapshot.The term clone life cycle management means generating a clone,refreshing a clone and deleting a clone based on user defined policiesand requirements. Refreshing a clone means deleting an existing clone,generating a new snapshot and then creating the clone again.

RBAC module 156 stores information regarding different clients/entitiesthat are given access to storage. For example, a particular businessunit may be allowed to read certain storage volumes and may not beallowed to backup, clone, replicate or delete any storage volumes. RBACmodule 156 manages the various roles and access type for differentapplications that may be executed in different host systems/computingenvironments, as described below in detail.

In one aspect, RBAC module 156 includes an authentication andauthorization module (see 402 and 406, FIG. 4A). User authentication mayhappen at multiple end points, for example, via a GUI login, a login APIfor clients or plugins and others. The authentication moduleauthenticates users against different domain/subsystem requirements, forexample, an Active Directory, a local Windows machine host system, openLDAP (lightweight directory protocol) and others. Once a user isauthenticated, an encrypted token is generated based on userinformation. In another aspect, a hash token is generated based on thegenerated token. The hashed token is saved at database 176. The hashedtoken may be based on MD5 (Message Digest Algorithm, 5, SHA (secure hashalgorithm)-1 or any other technique.

When the authenticated user logs back in, the user passes the token andthe SMS 132 decrypts the token, validates the token and regenerates thehash value. The hash value is compared with the stored hash value indatabase 176.

In one aspect, the authorization module of the RBAC module 156 createscustom roles (for example, a SMS administrator, backup administrator,backup operator, backup viewer, restore administrator and others),modifies existing roles, assigns and unassigns permissions to and from arole (for example, a dataset, policy, host, storage connection, adashboard, a report, discovery, remote installation and others), assignsand unassigns users to roles and assigns and unassigns resources (forexample, hosts, datasets, policy and others).

In one aspect, roles determine a set of capabilities that are availableto members of the role. For example, a backup administrator may beallowed to add a host system, install plugins, create a dataset, createa backup dataset, delete a dataset, create policies, delete backups,restore applications and others. A backup operator may be allowed tostart and stop existing dataset jobs, monitor backups, view backupreports and perform application level restore operations. A backupviewer may be given read only access to backups, view existing backupsand review job session details. A restore administrator may be allowedto perform restore operations using existing backups. The adaptiveaspects described herein are not limited to these roles.

In one aspect, once a user is authenticated, the RBAC module 156performs the following tasks: obtains user information from the token,checks the role that is assigned to the user; checks the permissionsassociated with the role and then either allows user access or sends anerror message if authorization fails.

In one aspect, storage 120 credentials are assigned to a particular userand maintained by both SMS 132 and SAL 151A (FIG. 1D). SAL 151A usesuser credentials based on the user token generated by the RBAC module156.

In one aspect, SMS 132 ensures that a logged in user token flows to anapplication for discovery, backup, restore and cloning operations. Whena logged in user does not have permission for an operation at theapplication level, then the application plugin reports that to SMS 132.The resources may then be locked and the user is notified. Details ofusing RBAC module 156 are provided below.

All the tasks conducted by the SM module 134 are organized and monitoredby the job manager 182. The job schedules are managed by the scheduler160. When a new job arrives, the job manager 182 stores the jobinformation in a database (for example, 176) with other existing jobs.The job manager 182 creates sub-tasks for executing the appropriate workflows. The sub-tasks depend on the nature of the job (for example,backup, restore, cloning or others). The job manager 182 updates thestatus of each task to provide real-time updates via a GUI, as describedbelow in detail.

The remote installation module 166 downloads an appropriate plugin foran application that is executed by a host system. Details regarding thefunctionality of the remote installation module 166 are provided below.

In one aspect, the policy data structure 170 is used to store policesfor different stored objects (for example, databases, data structures,VMs, storage volumes and others). The policy information is configurableand may be changed by a user. In one aspect, the policy data structure170 format is the same across different applications. This enables SMmodule 134 to manage storage services across different platforms withdifferent requirements and operating parameters, as described below indetail. The policy data structure 170 includes various policy objects tostore various policies each with a set of attributes that can be appliedto any dataset. The policy object stores a policy identifier, a policyname, description, a backup schedule policy, a retention count as to howlong a backup is to be retained, a replication policy to replicate abackup from one location to another, types of backups, applicationconsistent backups, and verification policies to verify a backup andothers. It is noteworthy that a policy object may be shared acrossmultiple datasets for multiple applications/plugins.

The reports module 172 is used to provide reports to users. The reportsmay be for different applications and in different formats. An exampleof different report types are provided below in detail.

In one aspect, the SM module 134 maintains the dataset 164 for differentapplications and application objects. Each dataset is uniquelyidentified and named. The dataset format for managing replication fordifferent applications is the same, regardless of how the applicationshandle information. A dataset may be associated with a policy datastructure that defines how an object is to be protected. The datasetformat used by the various aspects of the present disclosure allows auser to add or remove stored objects that need to be protected.

Dataset 164 is described below in detail with respect to FIG. 2E. In oneaspect, dataset 164 represents a container of application objects whereprotection attributes may be defined in terms of backup policy,replication profiles and retention policies that are all described belowin detail. Dataset 164 is a basic unit that is used to manage backupoperations. A user can add any permissible resource to the dataset frommultiple host systems/applications.

The database access layer 174 saves information in the database 176. Thedatabase 176 may be used to store information that is discovered by thediscovery module 150, policy information, host information, datasets andother information.

In one aspect, the database 176 may store various data structures formanaging the storage services and providing status to users. As anexample, the database schema for the database 176 is application formatindependent and may include various data structures to identifydifferent host systems to specific login information, a backup metadatastructure for storing information regarding backups, a data structure tomap backups to stored objects including VMs, databases and others, adata structure for storing information on backup types, i.e. applicationconsistent, full backup, copy backup, log backup for identifying volumesthat are involved in each backup, a data structure to track various jobsthat are managed by the job manager 182, discovery objects for storinginformation that is discovered by the discovery module 150, policy datastructure 170, storage footprint and storage resource information, adata structure for storing information regarding various plugins, roles,role attributes, storage connections and user information, includingcredentials.

In one aspect, SAL 151 stores information regarding the various storageresources that are used and available for different hosts. SAL 151maintains a “storage footprint” for different storage resources (forexample, storage systems including storage devices) used by differentapplications as described below in detail.

In one aspect, HAL 153 is used to communicate with another plugin (forexample, the VSC plugin 185A) that is used to collect informationrelated to storage used by different virtual machines, as describedbelow in detail.

SMcore Layer 130:

FIG. 1D shows an example of the SMcore layer 130A APIs (178) that caninterface with SMS 132, according to one aspect. The SMcore layer 130Aincludes a protection module 182, a recovery module 184, a discoverymodule 186, a plugin management module 188, a disaster recovery module190, a scheduling module 192, a provisioning module 194, a VSS (volumesnapshot services) requestor 196, a log backup module 198, a cloningmodule 199, a backup verification module 197 and SAL 151A.

SAL 151A may be used to communicate with the storage system 120. SAL151A maintains a storage footprint for each application. SAL 151Ainterfaces with each plugin to obtain storage resources that are managedby storage system 120 and made available to different applications 126(FIG. 1A). Details of using SAL 151A are provided below. In one aspect,SAL 151A uses ZAPIs (Zephyr Application Programming Interface) to sendand receive data from storage system 120.

In another aspect, the SMcore layer 130A interfaces with storage system120 via a storage system interface 195. An example of storage systeminterface is SnapDrive provided by NetApp Inc. (with derogation of anytrademark rights of NetApp Inc.).

The SMcore layer 130A interfaces with different types of plugins, forexample, a snap manager for exchange (SME) plugin 193A, a snap managerfor SQL (SMSQL) plugin 193B, a snap manager for hypervisor (SMHV) plugin193C (for example, plugin 128A and VSC plugin 185A) and others. The termplugin as used herein means a hardware processor executable layer thatis customized to interface with specific applications. For example, theSME plugin 193A is aware of Exchange format and behavior and providesExchange specific information to a module of the SMcore layer 130A.Similarly, the SQL plugin understands a SQL database application and itsobjects and provides that information to the core layer 130A.

The SMHV plugin 193C is able to interface with the hypervisor layer andprovides information regarding virtual disks, virtual machines that usethe virtual disks and the underlying storage information used by thevirtual disks.

In one aspect, the discovery module 186 manages discovery of plugins andthe resources that are used by each plugin. A consistent format is usedfor maintaining plugin information. For example, a standard schema isused to manage plugin information. The schema includes a unique pluginidentifier (PluginID), a plugin name, a plugin version, a plugin installpath, a description, a vendor name that provided the plugin, date aplugin was created and modified and a URL. The schema in conjunctionwith the object format (referred to as SMobject) described below allowsdiscovery module 186 to obtain plugin information and provide it to SMS132. To discover resources, the SMS 132 interfaces with the SMcore 130.The discovery module 186 of the core communicates with the respectiveplugin and obtains plugin information, as described below in detail

The protection module 182 is used for applying policy for backupoperations at a host system level. The recovery module 184 allows theSMcore layer 130A to recover or restore a stored object from a snapshotcopy. The plugin management module 188 is used to interface with SMmodule 134 to receive the proper plugins. For example, if a host systemonly uses Exchange, then the host system may need the SME plugin 193A.The plugin management module 188 provides the application information tothe SM module 134 so that the appropriate plugin can be provided.

The disaster recovery layer 190 coordinates disaster recovery with SMmodule 134 and the storage system 120, when there is a disaster and asnapshot needs to be recovered from a remote, secondary site whereinformation is replicated.

The scheduling module 192 is used for coordinating backup, restore andcloning related operation schedules, as described below in detail.

The provisioning module 194 is used to provision storage via a GUI. Theprovisioning module 194 provides information to the SM module 134 thatpresents the GUI to a user. In another aspect, the GUI may be presentedby host system 102A and the provisioning module 194 provides provisionedinformation to the SM module 134.

The VSS requestor module 196 (shown as VSS requestor) requests snapshotsin a Windows Operating system environment. The VSS requestor 196 theninterfaces with VSS service 191 that interfaces with the storage system120 to execute the snapshot operations.

The backup verification module 197 verifies a backup or replicationoperation. Since different applications may have different requirements,the backup verification module 197 facilitates the verification fordifferent applications, as described below in detail.

The cloning module 199 that assists in cloning a snapshot and a logbackup module 198 that assists in backing up logs. As an example, adatabase application may maintain a log to track changes to a databaseand the log backup module 198 assists in backing up those logs.

FIG. 1E shows an example of SMS 132 with the SM module 134 having acollocated SMcore layer 134A for interfacing with different plugins. Forexample, a Windows based host system 102B with SMcore layer 130Binterfaces with the SMS 132. The Windows host 102B includes a WindowsSnap manager plugin 187A and a file system plugin 195A. The Windows Snapmanager plugin 187A interfaces with Windows based operating systems orapplications for providing storage services through SMS 132. The filesystem plugin 195A is provided for a Windows based file system used formanaging data objects in a Windows operating environment. Plugin 195Ainterfaces with the storage system interface 195 to communicate with thestorage system 120.

The hypervisor (or host) 102A includes a virtual storage plugin 185A anda hyper-v plugin 185B (similar to 193C, FIG. 1C). The hyper-V plugin isused in a Hyper-V virtual environment. The virtual storage plugin 185Ainterfaces with the storage system 120 to obtain storage informationassociated with virtual storage that is presented to virtual machines.The hyper-V plugin 185B is used to interface with the Hyper-V layerexecuted within host 102A. This allows the SMcore layer 130A to developa storage footprint involving virtual and physical storage in a virtualmachine environment.

A UNIX host 102C includes a plugin 189A that understands an Oracledatabase (without derogation of any third party trademark rights) and afile system plugin 189B that understands the UNIX file system. Theplugins interface with the SMS 132 for protecting databases in a UNIXenvironment. For example, plugin 189B interfaces with the storage systeminterface 195 to communicate with the storage system 120.

FIG. 1F shows an example for providing plugins for differentapplications, including Exchange, Sharepoint, SQL, Windows, VSC, Oracle,Unix and Hyper-V using snapshot technology to protect information.

Format 200:

FIG. 2A shows a platform and application independent format 200 used bythe SM module 134 and the SMcore layers 130 for exchanging informationand then storing the information in database 176, according to oneaspect. In one aspect, regardless of which application data is beingprotected/managed, the system uses a request/response model format forobtaining and storing information regarding different applications usingthe various plugins. As an example, the objects used by format 200comply with the REST (Representational State Transfer) API model.

Format 200 shows a base class 202 for a snap manager request. The baseclass object is defined by its attributes that are described below. Thebase class includes a payload that determines what operation needs to beperformed. Object 204 is used to define the base class 202, which inthis case is a discovery request. Based on object 202, an interfaceobject 206 is defined that may be used for providing discoveryinformation.

Each SM request object includes a job object for a particular request.The job object may be used to show the details of a specific job,regardless of what application and environment is using a particularstorage service. For each response, there is associated a responseobject. This is shown as object 210, where the base class for theresponse is 208. The response interface, similar to request interface isshown as 212.

FIG. 2B shows an example of an object 213 (shown as SmObject) that maybe used for a request or a response, according to one aspect. Object 213is an abstract representation of an application object. Object 213format allows SMcore modules to communicate and interface with othercomponents in a format independent manner that does not depend on anapplication type. As an example, object 213 may be used for a response.Object 213 is based on details that are included within object 214 thatin this example, may be an Oracle database object.

SM object 213 includes various attributes. For example, object 213identifies a host system where the object or application may be running.The object has a unique identifier shown as object ID. Object 212includes a “key value” pair for the object. The key value pair may beused by a plugin and may include any information.

The object 213 may also include a unique name and a field that definesan object type. For example, the object type may be a database object,an object that provides information regarding virtual machines and otherobject types. The operations segment of object 213 is populated byobject 214.

In one aspect, object 214 may be used to represent any application type.This allows the SM module 134 to manage and protect informationregarding different application types, as described below in detail.

Storage Footprint:

FIG. 2C shows an example of storage footprint 215 that is maintained bySAL 151 at SM module 134 and/or SAL 151A, according to one aspect. SAL151 obtains information from SAL 151A of each SMcore layer 130 that isaware of each host system operating environment. In one aspect,information regarding different storage devices/objects is maintained ina standard format. This enables SM module 134 at SMS 132 to managestorage services related to different storage device types and storageprotocols, as described below in detail.

The storage footprint 215 may be stored as an integrated stored object216 that includes storage information for different storagedevice/protocol types. For example, object 216 includes informationregarding a CIFS share represented by object 218, a host file systemthat is backed up in a SAN storage represented by object 220, a virtualdisk resource 222 that is deployed in a virtual machine environment, aNFS based storage represented by object 224, a storage system LUNrepresented by object 226, and a NFS and CIFS system level directoryrepresented by object 228 that is represented by object 228. By managingstorage resources that are used by disparate applications andenvironments, SM module 134 can efficiently manage storage services forsuch applications and environments, as described below in detail.

SAL 151 abstracts storage information and stores the storage informationas object 215. This allows SM module 134 to replicate, clone and restoreinformation regardless what storage system type, storage device type orfile system protocol that is used for storing information.

In one aspect SAL 151 (and 151A) may be implemented as an API that isused for backups, restore, cloning and provisioning operations fordifferent applications, without having the applications getting involvedwith the semantics of storage system 120. SAL 151 abstracts storagespecific technology decisions, for example, whether to use snapshot orsingle instance storage (SIS clone) for backup and mounting, asdescribed below in detail.

FIG. 2D shows an example of a storage footprint object 216. Object 216is based an object 230 that provides information regarding host systemstorage resources, storage system resources, virtual machine environmentstorage resources and others. It is noteworthy that object 216 format issimilar to the format 200 described above in detail.

In one aspect SAL 151 (and SAL 151A) are used for volume management, LUNmanagement, initiator group (“igroup”, where an igroup identifies a listof initiators (or adapter ports) that are allowed to access a LUN or astorage volume) management, snapshot management, mounting snapshots,clone management, replication management and other operations.

For volume management, SAL 151/151A may be used to retrieve volumeinformation from storage system 120, provision a volume, create a volumeclone, set a state for a volume and dismount a volume. For LUNmanagement, SAL 151/151A facilitates creating a new LUN, delete a LUN,retrieving LUN information, mapping a LUN to an igroup, retrieving LUNmapping information, getting LUN attributes, setting LUN attributes,getting LUN details including LUN size, enabling LUN resizing, taking aLUN online or offline, getting a LUN path information, creating a targetLUN, renaming a LUN and other information.

Igroup management, includes getting igroup information for a LUN/storagesystem, adding initiators to an igroup, binding an igroup to a port set,renaming an igroup and removing an igroup.

Snapshot management includes creation of a snapshot, removing asnapshot, restoring a snapshot, cloning a directory, performing a filebased restore or volume based restore, getting a list of snapshots for avolume, renaming snapshots, mounting a LUN from a snapshot anddismounting a LUN from a snapshot.

Replication management includes setting SnapMirror/Vault information,getting replication status, setting policy rules for replications,removing a retention policy and other operations.

Dataset 248:

FIG. 2E shows an example 248 (similar to 164, FIG. 1C) for maintaining adataset, according to one aspect of the present disclosure. As mentionedabove, a dataset is an abstract representation of a container forapplication specific objects for executing a storage services relatedoperation, for example, a backup operation, a clone life cycle operationand others. Briefly, a dataset is an independent unit for defining andmanaging backup operations. A user can add resources from multiple hostsand applications that are managed by SM module 134 to a dataset.

The dataset may be used to define protection attributes as defined by abackup policy, a retention policy that defines for how long a backup isretained and replication profiles and others. Examples of protectionattributes include, defining when a backup is taken, the type of backup(i.e. full backup or a selective backup of log files); update policythat defines when the backup is updated; retention count may be definedas older than a certain number of days and count based i.e. after acertain number of backups, backups are deleted. The attributes may alsobe used to define if there needs to be a specific verification for thebackup, for example, one dictated by SQL and Exchange servers.

In FIG. 2E, the high-level policy examples are shown as object 250.These policies may be applied for any application. The application levelor specific policy is shown as object 252 that is based on objects 254and 256. For example, object 254 defines the policy parameters forbacking up an Exchange server. Object 256 defines the policy forverifying the backup. Object 258 defines the backup schedule that ismaintained by the scheduler 160 of SM module 134. By managing storageservices via a standard dataset format, SMS 132 is able to efficientlymanage backup, clone, restore and other operations for different hosts,VMs, applications and plugins. Example of generating datasets isprovided below in detail.

FIG. 2F shows a process 251 for generating a dataset, according to oneaspect of the present disclosure. The process begins in block B253, whena host system is discovered. The host may be discovered by SMS 132 oradded via a user interface. As an example, hosts within a subnet may beautomatically discovered by sending a discovery packet. SMS 132 storeshost information as part of database 176. Each host is uniquelyidentified and if the host is part of a group, then the host group isalso uniquely identified. SMS 132 also stores a network access address(for example, the IP address) that is used to access the host, a portidentifier used to connect with the host, a cluster node identifier, ifthe host is a part of a cluster, a description of the host, and the dateit is created and modified. This schema is consistent for all hosttypes.

The SMS 132 communicates with the host to determine if a SMcore layer130 is installed. When the SMcore is already installed, then theappropriate SMcore layer 130 discovers the associated plugin. If theSMcore layer is not installed, then it is installed with one or moreplugins.

In block B257, SMS 132 discovers the resources used by the plugins viaSMcore layer 130. Details of the application resource discovery areprovided below.

In block B259, the user is presented with a dataset authoring tool. Anexample of the authoring tool is provided in FIG. 8D, described below indetail. In block B261, the user selects an entity (for example, adatabase) that it wants to protect. The entity may be all the dataassociated with an application or a sub-set of the data.

In block B263, a dataset is generated and associated with a policy. Asdescribed above, the dataset includes core policy attributes (forexample, 250, FIG. 2E) and application specific policy (for example, 254and 256, FIG. 2E). The process then ends in block B265.

Plugin Installation:

FIG. 3A shows a process 300 for installing a plugin, according to oneaspect of the present disclosure. The process begins in block B302, whena host system is initialized and operational.

In block B304, the SMS 132 is installed. The SMS 132 may be installed asa virtual machine at a computing device within a virtual machineenvironment. In block B306, the SMS 132 first determines the hostconfiguration. The host configuration information may be obtained fromthe SMcore 130, when the SMcore is installed at the host.

Based on the appropriate host configuration, in block B308, theappropriate plugin is pushed to the host system by the remote plugininstallation module 166. Thereafter, the plugin is installed at the hostsystem.

Resource Discovery:

FIG. 3B shows a process 310 for determining host and applicationresources by SMS 132 from different applications, storage system 120 andother modules of system 100, according to one aspect. The discoveryprocess uses the request/response format described above with respect toFIGS. 2A-2B such that discovery is not affected by individualapplication formats/requirements.

The discovery is initiated in block B314 by SMS 132 and based on alogged in user, a user token is generated. In one aspect, the token is aunique identifier for a specific user and for a specific discoveryoperation. The token is then used across different platforms forobtaining information. In one aspect, the RBAC module 156 performsauthentication for the user and generates the token. The token may beencrypted and provided to the user. The RBAC module 156 stores a hashvalue for the token at database 176 for future authentication.

In block B316, SMS 132 determines if an instance of the SMcore layer 130is installed at a host system it is communicating with. This may bedetermined by the discovery module 150 that sends out a discoveryrequest. If a response is received by the discovery module 150, then SMS132 assumes that the SMcore layer 130 is installed.

If the SMcore layer 130 is installed, then in block B318, the SMcore 130discovers the appropriate plugin that is being used by the host system.The plugins are discovered by the SMcore layer 130 and provided to SMmodule 134. In one aspect, SMS 132 stores at least the followinginformation regarding each plugin: a unique plugin identifier (PluginID,a plugin name, a plugin version, a plugin install path, a description, avendor name that provided the plugin, date a plugin was created andmodified, a URL, a host-plugin relationship, a host identifierassociated with the plugin and the date the plugin information objectwas created.

If the SMcore layer is not installed, then in block B320, the SMcorelayer 130 is installed at the host system. The appropriate plugin isalso installed. The type of plugin will depend on the host systemapplications. For example, if the host is a Unix system managing anOracle database, then the plugin is the Oracle plugin 189A. Similarly,for a Windows environment, the plugin is Windows Snap manager plugin187A. For a virtual environment, plugins 185A and 185B are installed.The installed plugin are discovered in block B322.

In block B324, the user token is provided to the SMcore layer 130. TheSMcore layer 130 then passes the token to the installed plugin(s). Theplugins then pass the discovery request to the appropriate application.For example, if the request involves virtual machine storage resources,then the hypervisor plugin will request the storage resources from thehypervisor and the virtual storage plugin will request the resourcesfrom the storage system. This allows the SMcore 130 to obtain detailsregarding both the virtual and physical storage. The discoveredresources are then provided to SMS 132 in block B326. Database 176 isthen updated to store the discovered information. In one aspect,regardless of the application type, the data is stored in the formatshown in FIG. 2C and described above.

FIG. 3C shows an example of a process 330 showing resource discoveryinvolving different applications, plugins and the SMS, according to oneaspect of the present disclosure. As an example, the process begins inblock B332, when a discovery request is received by the SMS 132. Therequest may be generated from a GUI or a CLI. In block B334, the SMS 132generates a unique user token for the discovery request. In block B336,the token with the discovery request is sent to the appropriate SMcorelayer 130. The request format is the same regardless of what plugin orapplication is involved.

In blocks B337 and B342 the SMcore layer 130 sends a discovery requestto the Oracle application plugin 189A and the file system plugin 189B,respectively, when storage is not virtualized. The application plugin189A provides a list of files that need to be discovered in block B340.The file system plugin provides the storage footprint for the list offiles in block B344.

When storage is virtualized, as determined in block B345, a discoveryrequest is sent to the VSC plugin 185A (or any other hypervisor pluginthat may be appropriate for the virtual environment). The pluginconducts discovery in block B348 to obtain both virtual storage andphysical storage information and provides the same to SMS 132. In blockB350, the storage information is then stored at database 176.

FIG. 3D shows an example of a process 352 for discovery resources in aWindows based host (for example, 102B (FIG. 1E)). The process flow 352is similar to the process 330 described above with respect to FIG. 3C. Adiscovery request is received by SMS 132 in block B354. A user token isgenerated in block B356 and a discovery request is generated for theSMcore 130 in block B358. The SMcore 130 creates a job in block B360 andretrieves the user token in block B362 and thereafter, initiates adiscovery request for the Snap Manager plugin 187A that retrieves theuser token in block B366. The application based discovery is initiatedin block B368, while storage resource discovery request is initiated inblock B370. The storage resource information is obtained in block B372.

In blocks B374 and B376, the plugins update any modified state to theSMcore 130 that provides the information to the SMS 132. For example, ifa new database is added to an existing SQL instance, then blocksB374/B376 allows the SMS 132 to reflect the status of the newly addeddatabase to a user via a GUI. Also if a database is deleted, or adatabase property is changed (e.g. a new file group is added) that isalso reflected immediately in the SMS 132 database 176 and then the GUI.This is enabled because SMcore 130 subscribes to events for anyinventory related changes that occur with respect to each plugin. Usingthis mechanism SMS 132 inventory information is automatically updatedwith real time changes happening on a remote application. Thereafter, adatabase update request is sent in block B378 and the database isupdated in block B380.

FIG. 3E shows a generic example of host/inventory management using SMS132, SMcore 130, plugins 128A-128N (see FIG. 1A) and ZAPI interface usedby SAL 151A and the storage system interface 195, according to oneaspect. The process begins in block 360 when a role is requested by auser. SMS 132 returns a role 362, based on user information stored atdatabase 176. In block 364, the user connects with the role and SMS 132returns a user instance.

In block 366, the user requests information regarding a host group. SMS132 returns the information in block 368.

In block 370, host information is requested. A discovery process istriggered in block 372 and returned in block 374. In this example, a WMI(Windows Management Interface) plugin is used to obtain the information.The host information is then returned in block 376.

In block 378, remote install process is triggered. SMS 132 communicateswith SMcore 130 in block 380. SMcore 130 then manages the plugin installprocess in block 382. The installation instance is provided to SMcore130 in block 384 and returned to SMS 132 in block 386 and then returnedto the user in block 386A.

A storage connection is established in blocks 388 and 390. As anexample, a CIFS share is created in blocks 392, 392A, 392B, and 392C. Avolume is created in blocks 394A, 394B, 394C and 394D. It is noteworthythat a volume may be created by other management application(s) (notshown). A storage disk is then created in blocks 396A-396E.

RBAC Architecture and Process Flows:

In one aspect, SMS 132 is configured to manage RBAC requirements forvarious environments, including SQL, Oracle database, active directory,Open Lightweight Directory Access Protocol (LDAP), Windows operatingenvironment and others. During SMS installation, a user is asked toenter SMS administrator credentials with relevant permissions. Once theuser is able to login as an administrator, it can assign other users todifferent roles, as described below in detail.

Each role has a set of attributes and a storage connection is associatedwith one or more users. One or more hosts are also associated with theuser. Each SMcore 130 (and SMS 132) implements a local cache of RBACroles and connection mappings which are queried by SAL 151A forretrieving storage connection for a specific user, as described below indetail.

FIG. 4A shows a block diagram of an architecture 400 used by the RBACmodule 156 (FIG. 1C) for managing roles and access within system 100,according to one aspect. Various service points within system 100require proper authentication and authorization. For instance, SMS 132,the various plugins and storage system 120 needauthentication/authorization for users to either access storage or beable to perform storage related services. The parameters forauthentication/authorization maybe different when a storage servicerequest is being processed from an application to the storage system'sperspective. For example, the RBAC rules and semantics may be differentfor hosts' 102A-102C and storage system 120 (See FIG. 1E). A UNIX host102C may have different authorization and authentication requirementsvis-à-vis Windows host 102B or the Hypervisor 102A. Similarly, thestorage system 120 may have different requirements for authenticatingand authorizing access to storage space and stored data containers. Thearchitecture 400 enables SMS 132 to manage authentication andauthorization roles for the various layers described above.

The SMS RBAC module 156 includes an authentication module 402 and anauthorization module 406, according to one aspect. The authenticationmodule 402 provides authentication services to authenticate users. Theauthentication may be based on system 100 settings.

The authorization module 406 provides authorization based on a roledefined for the user, a permission name associated with the role, anoperation type specified for the role/user, as well as an object typethat the user intends to access.

As part of authentication, authentication module 402 generates a uniquetoken after a user name and a password is authenticated. The token maybe based on the user name, current date time and a default validity, forexample, for 24 hours. The token may be encrypted using AES (AdvancedEncryption Standard) based encryption and then passed to the client. Theclient may then start communicating with different modules using thesame token.

The RBAC module 156 maintains domain specific parameters 412A-412N fordifferent domains, for example, active directory, open LDAP, and others.This allows RBAC module 156 to accommodate authentication for differentdomains.

The RBAC module 156 also interfaces with a storage RBAC module 408 viaSAL 151. The storage RBAC module 408 maintains RBAC information asrelated to the storage devices and storage objects maintained by thestorage systems.

The RBAC module 156 further interfaces with an application RBAC module414. The application RBAC module 414 may interface with applicationspecific RBAC modules, for example, hypervisor RBAC module 416, WindowsRBAC module 418, database RBAC module 420 or any other application basedRBAC module 422. The different applications may have different RBACrequirements and RBAC module 156 manages these different requirements asdescribed below in detail.

FIG. 4B shows a format 424 used by RBAC module 156 to manage varyingRBAC requirements. Format 424 maps user roles to storage connections,role name and a role identifier. The storage connection may be based onone or more virtual servers (VServers) credentials.

A user having credentials 432 is part of a group 428 that belongs to aspecific domain 426. An example of a domain may be a SQL databasedomain, an Oracle database domain, a vCenter domain and others. The usercredentials include storage connection credentials and the user specifictoken described above. In one aspect, user credentials 432 may be storedat a data structure by SMS 132 that keeps track of user to credentialmapping for supporting RBAC requirements for different domains.

Each user is assigned a role 434 that provides certain privileges. Everyuser has one or more roles. For example, a user may be a backupadministrator 434A, a backup operator 434B, a backup viewer 434C, arestore administrator 434D, and any other user defined role 434E becausea user may create new roles, modify existing roles or delete roles basedon user needs and the operating environment. Examples of these roleshave been described above.

Each role is associated with certain permissions 436 that has certainattributes 438. The permission allows the user to execute certainoperations (440) on an object 442. The permissions may allow a user toaccess, read, modify or delete specific objects or informationpertaining to the various objects mentioned herein. Examples of objects442 are shown as a host object 442A, a VM object 442B, a plugin 442C, adataset 442D, resources (for example, storage resources 442E) and apolicy object 442F.

The host object 442A may have configuration information regarding a hostas well as stored data objects for a specific host. The VM object 442Bmay have configuration information regarding a VM, the resources used bythe VM and data containers stored using the VM. Dataset 442D has beendefined and described above. Resources 442E may include informationregarding storage devices, network resources and others. Policy 442F hasalso been defined and described above in detail.

In one aspect, SMcore layer 130 implements a local RBAC cache to cacheroles and storage connections for a specific user for a host system.This allows the SMcore layer 130 to determine user rights andpermissions for backup restore, cloning and other storage services.

FIG. 4C shows a process flow 450 for using the RBAC module 156 and thearchitecture of FIG. 4A as well as the format of FIG. 4B, according toone aspect of the present disclosure. The process begins in block B452,when a user needs to be added to use SMS 132 features or any other time.The process blocks are executed using a GUI or a CLI.

In block B454, a user token is generated for accessing a stored object,for requesting a storage service related to the stored object or for anyother reason. This may require user input. The RBAC module 156 keepstrack of user credentials in a data structure (432) for supportingapplication specific RBAC and authentication schemes. This enables SMS132 to map user and application specific credentials.

In one aspect, the user may use a username and a password to login toSMS 132. The user name can be an active directory user, a LDAP user,local Window machine user or any other type.

In block B456, the user is assigned one or more roles with specificattributes. The user may have administrative rights that enables theuser to create roles. As described above, a role is a set of pre-definedoperations a user can execute on an object. In one aspect, a role is acontainer with certain capabilities that are defined by role attributes.As an example, the attributes include “view dataset”, edit dataset,create data set, delete data set, create storage connection and deletestorage connection.

In block B458, the user is assigned to a host (or a host group), where ahost group has more than one host system. In block B460, the user isassigned a storage connection. As an example, a storage connection maybe through a virtual server to access a storage volume.

In one aspect, a standard format is used to manage storage connectioninformation, which may be stored as a storage connection object indatabase 176. Regardless of who uses storage, the storage connectionobject is maintained in the same format. For example, the storageconnection object format includes, an identifier of a storage device,name of the server managing the storage device, IP address of theserver, a port identifier for accessing the server, password, roleidentifier for the storage connection, date the storage connectionobject was created and modified, when applicable.

In one aspect, the foregoing RBAC architecture and layout allows SMSserver 132 to manage RBAC functionality across different platforms,applications and operating environments through a single pane, asdescribed below in more detail with respect to FIGS. 8A-8T.

Backup Process Flows:

In one aspect, SMS 132 and SMcore layers 130A-130N with the appropriateplugin(s) are used to backup information stored for any applicationtype, using storage system 120. The backups are initiated by a backupschedule for one or more stored objects or on-demand based on userinput. The SMS server 132 creates a backup job in a standard format sothat backup job for every application can be tracked and reported to theuser. Once the job is created, the SMS server 132 notifies the SMcore130 of the host system that executes the application for the storedobjects.

The SMcore 130 at the host then drives the backup process flow by firstdiscovering the appropriate resources used by the application and thestorage system 120. Details regarding the process flows are providedbelow.

FIG. 5A shows a process 500 for creating a dataset and then executing astorage service operation, for example, a backup operation using SMS132, according to one aspect. The process begins in block B504, when adataset is generated based on user input. An example of a dataset isshown in FIG. 2E and described above in detail.

In block B506, an application object is added to the dataset. Theapplication object may be a database, a virtual machine, a datacontainer or any other object type. A protection policy is applied tothe dataset in block B508. The protection policy includes a retentionschedule (i.e. for how long a snapshot or backup) copy of theapplication object is to be retained. A backup schedule that defineswhen and how often the object is to be backed. The protection policyalso defines a backup type, for example, a full backup of a storagevolume or a log backup. It is noteworthy the dataset is managed in thesame format regardless of the object type, the application type and/orbackup type.

In block B510, the SM module 134 at SMS 132 initiates a backupoperation. The backup operation is based on the policy for one or morestored objects that are defined in the dataset. The job manager 182generates a job object to track the backup operation. Thereafter, inblock B512, the SMcore layer 130 associated with the host system for thestored object is requested to perform the backup operation. Because theSMS 132 and SMcore layer 130 use a standard request and response formatto communicate as described above with respect to FIG. 2A, SMS 132 isable to initiate and manage backup operations for different host systemsand applications.

In response to the backup request, in block B514, the SMcore layer 130initiates a discovery operation for a backup operation. In one aspect,the discovery request is routed to an appropriate plugin depending onthe object and the dataset. FIGS. 3C and 3D described above showexamples of the discovery process. For example, if the object is a SQLdatabase, then the discovery request is forwarded to the SM plugin 187Aexecuted by the appropriate host identified by the dataset. For a UNIXhost, the Oracle plugin 189A is used to discover the list of files forthe database.

The storage footprint for the database is also obtained. The storagefootprint may be obtained using SAL/SDW plugins for the Windows case.For a virtual environment, the Hypervisor plugin and/or the VSC pluginmaybe used to obtain the storage footprint. Once the storage footprintis obtained, in block B516, the process groups the storage objects intomultiple sets to take a backup. This reduces the number snapshots thatthe system has to take.

In block B518, the plugin for the backup operation is quiesced i.e.placed in a state where other operations are delayed until the backupoperation is completed. SAL 151A is instructed to take a snapshot inblock B520. Once the snapshot is taken by the storage system 120 usingstorage services module 122 (the details of which are not providedhere), the SMcore 130 instructs the plugin to unquiesce in block B522.Thereafter, the SMcore 130 obtains the backup metadata in block B524. Anexample of backup metadata is shown in FIG. 5D, described below indetail. The backup metadata is stored by the SMS 132 for restoreoperations described below in detail. In block B526, the retention andreplication policy for the backup is implemented. This will depend onthe policy itself. In block B528, the SMS 132 registers the backup andstores the metadata for restore operations. Thereafter, the process endsin block B529.

FIG. 5B shows an example of executing the process 500 from block B510onwards in a virtualized environment where VMs are used or whereapplications within a VM environment is used. A backup request isreceived by SMS 132 (shown as B531). The backup job object is created bySMS 132 in block B510. The backup request to SMcore 130 is sent in blockB512. The discovery takes place in block B514. The databases are groupedin block B516. The rest of the process blocks are self-explanatory,including blocks B528A/B528B for applying retention and replicationpolicies

FIG. 5C shows portions of process 500 (from block B510 onwards in anon-virtualized environment, after a request for a backup is received inblock B531. The process blocks of FIG. 5C are similar to FIG. 5B, exceptthe storage footprint also includes virtual disks and discovery ofvirtual disks.

FIG. 5D shows a format 530 for managing backup metadata for a pluralityof applications, according to one aspect. In one aspect, SM module 134maintains a same data structure format representing all applicationplugins. A common backup metadata interface is maintained for differentplugins, as described below. If a plugin wants to extend the metadata,then SMS 132 allows addition of application specific tables and links.The common backup metadata interface is used to collect and store themetadata. Although different applications may have different backupmetadata and metadata formats, the SM module 134 stores the metadata ina generic format, regardless of application type. This makes the overallprocess for managing the backups and the associated metadata efficient,as described below in detail.

FIG. 5D shows a main backup object 532 that is used to store a backupidentifier (nsm_backup_id) and a backup name. The backup object 532 mayalso be used to store the time the backup was created and when it waslast modified. A job identifier (job_id) identifies the job that wasused to create the backup. A protection identifier (protect_group_id)defines the dataset to which the backup belongs. The protection groupidentifier is associated with a protection group object 236 thatincludes details regarding the protection group i.e. the protectiongroup name, when it was created, modified and a description.

Object 532 is also associated with object 534 that is labelled asnsm_sm_object. Object 534 identifies the object that is being backed up.The object is uniquely identified by a name and an identifier, includinga global unified identifier (GUID). Object 534 also indicates the objecttype, for example, a virtual machine, a database and others and includesan identifier for a snapshot instance. Object 534 is associated withobject 540 that identifies what is being backed up. Object 540 alsoidentifies a metadata map and a backup metadata.

Object 532 is also associated with a protection group object 538. Theprotection group object is identified by a protection group identifier,group name, description and the time the group was created and/ormodified.

The backup metadata is shown in object 536 that provides the databaseschema for handling metadata. Object 536 represents the common aspectsof backup metadata across multiple applications.Applications/application plugins may customize the metadata which allowsSMcore modules to interact in a seamless manner, regardless ofapplication type.

Object 536 provides a mechanism to attach and lookup attributes ofbackups along with specific metadata elements. Object 536 identifiesbackup metadata with an identifier. The metadata also identifies apolicy and the backup for a data container. The object that is backed upis identified as nsm_sm_object_id. An application specific key valuepair may be stored in object 536 as metadata key and metadata value. Theplugin associated with the backup is identified by the backup plugintype. The backup sequence identifies the number of the backup when morethan one backup exist.

The generic backup representation of FIG. 5D uses key value pairs sothat different application requirements can be met efficiently. Thusbackup metadata for different plugins can be extended using thisapproach.

An example of object 534 is shown as SMbackup object 542. Object 542 isdesigned to abstract all the backup information including resources thatare backed up. Object 542 identifies the components that are backed up(“Backup Component”). The identifier for the backup and a backup name.The date of the backup and the backup type. The job identifier is shownas INt64. The metaobject is shown as the “keyvalue” that is customizedfor different applications. The policy name and the protection groupnames are also shown in object 542. If a plugin needs additional data orobjects, then that can be added under “Operations” showing “Smbackup”. Anew class may be used to represent plugin specific detailed objectinformation backed by using a plugin specific database schema.

FIG. 5D also shows an example of an interface 246 labelled as“ISMBackupRepository” that is used to obtain backup metadata. Theinterface may also be used to delete backups, get backups and registerbackups.

Restore Workflows:

FIG. 6A shows an example of a process 600 for restoring a stored objectthat has been backed up. The stored object may be a SQL database in aWindows environment. Process 600 shows interactions between SMS 132,SMcore 130, SMSQL plugin 193B and SAL 151A/SDW plugin (shown as SDW)195A (also referred to as file system plugin). The process begins when arequest to restore a stored and backed up object (for example, a SQLdatabase) is received in block 602. It is noteworthy that the sameprocess may be used with a different stored object stored for adifferent application. The restore request may be received via a GUI ora CLI.

In response to the request, SMS 132 (i.e. the recovery module 152 of SMmodule 134) obtains the backup details from database 176 in processblock 604. As described above, the backup metadata is stored in a sameformat regardless of object or application type and may be searched forusing a key-value pair, for example, a backup ID and a host ID or othervalues.

The job manager 182 then creates a job and an application recoveryrequest in process block 606. An example of the recovery request objectis shown as 606A. The recovery request object identifies the Smobject asa SQL database and also includes information on the snapshot and thelocation information for the snapshot files. The recovery object alsoidentifies the backup type.

In process block 608, a pre-restore request with object 612 is issued toSMcore 130 that issues the pre-restore request to plugin 193B thatinterfaces with the SQL application. Object 612 includes a list ofdatabase files that are being restored. The plugin 193B detaches thedatabase that is being restored into a restore mode in process block610.

In process block 614, the pre-restore request with object 616 is alsosent to SAL 151A/SDW 195A that interfaces with the storage systeminterface 195 and the storage system 120. Object 616 provides the listof files that are to be restored. SAL 151A/SDW 195A decides if therequest is for a single file or a complete LUN restore.

A restore request is sent to SMSQL plugin 193B with object 620 (similarto 612). A restore request is also sent to SAL 151A/SDW 195A via SMcore130 with object 624 that is similar to 616. The storage system 120restores the files and notifies SAL 151A/SDW 195A.

A post-restore process is triggered in process block 626. The plug-in193B is requested by SMcore 130. In block 628, the plugin 193B removesthe database from the restore mode and attaches the database so that itcan be accessed.

In process block 630, a post-restore request is sent to SAL 151A/SDW195A with object 632, similar to object 624. Thereafter, the processends.

FIG. 6B shows a process 634 for restoring a complete VM based storage,according to one aspect of the present disclosure. Various FIG. 6Bprocess blocks are similar to the process blocks of FIG. 6A, except inFIG. 6B, the virtual plugin 185A is also involved to obtain informationregarding virtual disks and the association of the virtual disks to theactual storage managed by the storage system 120. The example of FIG. 6Bis also for restoring a database that is being managed in a VMenvironment i.e. a SQL database in a VM environment (i.e. a virtualizedSQL).

In process block 636, a restore request is received. The request may bereceived via a GUI and identifies a backup that needs to be recovered.The request may also identify the virtual machine where the SQLapplication may be running. The request is received by SMS 132 and thenpassed to the recovery module 152. The recovery module 152 uses backupidentifier information from the request and obtains backup details fromthe stored backup metadata at database 176. An example of backupmetadata is shown in FIG. 5D and described above. The job manager 182then creates a restore job in block 640. An app recovery request objectis created by SM module 134 and is shown as 642. Details of 642 aresimilar to 606A described above with respect to FIG. 6A.

In process block 644, a pre-restore request is sent to the SMcore layer130 at a host system that manages the database that is being restored.The SMcore layer 130 presents the request to the SMSQL plugin 193B withobject 648. In block 646, the plugin detaches the database mentioned inthe recovery object 624 and 644 and places the database in a restoremode.

The pre-restore request is also sent to SAL 151A/SDW 195A in processblock 650 with object 654. Object 654 provides the list of files thatare to be restored and a snapshot locator i.e. where the snapshot islocated from the storage system 120's perspective. The SAL 151A/SDW 195Adetermines the type of restore operation, similar to process block 616,described above with respect to FIG. 6A.

The pre-restore request is also provided to the VSC plugin 185A in block656. The VSC plugin 185A is also provided with a list of virtual disksin object 658 that are used to store the database files.

In block 660, a restore request is first sent to the SQL plugin 193Bwith object 662 i.e. list of the database files. The restore requestwith list of storage system files (i.e. Snapshot locator and a list ofapplication files in object 666) is also sent to SAL 151A/SDW plugin195A in block 664.

In block 668, the restore request is sent to the VSC plugin 185A withobject 670. Object 670 includes virtual disks and a VSC backup objectthat identifies the backup from the VSC's perspective. The VSC plugin185A mounts the virtual disks in block 671. Thereafter, a post-restorerequest is sent via SMcore 130 to SQL plugin 193B in block 672 with alist of databases (see object 676). In block 674, the SQL plugin 193Bremoves the database from the restore mode and makes the databaseavailable for use.

The post-restore request is also sent to the SAL 151A/SDW plugin 195A inblock 678 with object 680. In block 682, the post-restore request isalso sent to the VSC plugin 185A with object 684, which is similar to607. The VSC plugin 185A then detaches from the VM in block 682A and theprocess ends.

FIG. 6C shows a process 631, similar to process 634, except the restoreprocess is for a partial virtual disk (or VMDK), according to oneaspect. The similar process blocks have been described above and forbrevity sake the description is not being repeated. For example, block633 is similar to 636, block 635 is similar to 638, block 637 is similarto 640 and object 639 is similar to 642.

Block 641 is similar to 644, object 645 is similar to 648 and block 643is similar to 646. Block 647 is similar to 650, object 651 is similar to654 and block 649 is similar to 652.

Block 653 is similar to block 656 ad object 661 is similar to object658. In block 655, the VSC plugin 185A mounts the virtual disk that areaffected by the restore operation and attaches itself to the VM. Object659 is returned to SMS 132 with information regarding the source virtualdisk and the destination virtual disk.

Block 663 is similar to 660 and object 663A is similar to object 662.

Block 665 is similar to block 664. Object 669 is similar to object 666but also includes destination virtual disk and a drive letter. In block667, the files that need to be restored are copied to the active filesystem.

In block 671A, a restore request is sent to VSC plugin 185A with object673. Object 673 includes all the information of object 670 and alsoincludes destination virtual disk and the assigned drive letter for therestored database.

Block 675 is similar to 672 and object 679 is similar to 676. Block 677is similar to 674, while block 681 is similar to block 678. Object 683is similar to object 673. Block 685 is similar to block 682, while block687 is similar to block 682A. Object 689 is similar to object 684 andalso includes a mapping of source virtual disk to the destinationvirtual disk where the database is restored.

Clone Life Cycle Management:

In one aspect, SMS 132 provides a centralized tool for cloning backupsand managing the life cycle of the clones. The term clone as used hereinmeans a “flexclone” volume that is created based on a clone of snapshotof a flexible storage volume i.e. a storage volume whose size can bevaried and managed by the storage system 120. When a flexclone volume iscreated it is associated with a parent storage volume and a basesnapshot. The adaptive aspects described herein are not limited to aflexclone or a clone of a flexible volume.

FIGS. 7A-7F show various methods and systems for cloning and clone lifecycle management, according to the adaptive aspects of the presentdisclosure. In one aspect, a user is able to select a backup using a GUI(as described below) and create a clone on-demand. Clone creation mayalso be triggered by a schedule or clone life cycle management. The sameis available for a clone delete operation, as described below in detail.

A user may set a clone lifecycle for one or more datastructures/databases using a GUI or a “cmdlet” (for example, aPowershell cmdlet used in the Windows environment (see 180, FIG. 1D)). Adataset is created and stored application objects (for example,databases are added to the dataset). An option to manage the clone lifecycle is selected from the GUI. A wizard then creates a clone lifecycledataset. A new policy is created (or an existing policy is used) andassociated with the clone dataset. Thereafter, the clone life cycle canbe initiated based on a preferred schedule.

In one aspect, SMS 132 enables clone lifecycle management for differentstored objects that are maintained by different applications. BecauseSMS 132 maintains information regarding backups and datasets in anapplication/platform independent format, it can centrally manage thelife cycle for various clones for different applications, including VMs.

FIG. 7A shows a process 700 for managing the life cycle of a clone,according to one aspect. The process begins in block B702. In blockB704, an inventory view is presented to a user via a GUI. The inventoryview (for example, the view shown in FIG. 8C) may present storedapplication objects, for example, databases to the user. The applicationobjects may vary from one application to another. The user selectsstored objects for cloning. It is noteworthy that the user may select alive database for cloning. Multiple databases may be selected at thesame time for cloning.

In block B706, a clone dataset is created for the clone and a clone lifecycle option is selected for the clone dataset. The dataset includeswhat will be cloned (for example, a cloned instance name, parametersettings, mount point settings and others). An example of creating adataset is provided above in detail.

Thereafter, in block B708, a clone policy is generated and associatedwith the clone dataset. An example of the clone policy is shown inscreenshot 724 of FIG. 7B. It is noteworthy that different applicationsand plugins will have different configuration settings based on theapplications and preferences for specific entities that are part of thedataset. The various settings are part of the dataset which makes itefficient to manage clone life cycles.

In block B710, a backup workflow is initiated. Examples of backupworkflows have been described above with respect to FIGS. 5A-5C. Oncethe backup workflow is completed, the SMS 132 receives the backupidentifier. Thereafter, a clone workflow is initiated in block B712.Based on the clone policy, the clone life cycle is also executed and theprocess ends in block B714.

FIG. 7B also shows an example of a clone dataset 716 that is associatedwith clone application objects 718 and stored at database 176. A clonepolicy object 720 is associated with the clone dataset 716. An exampleof the clone dataset with a clone preference 722 is also provided inFIG. 7B. An example of a clone policy 724 is also provided. In oneaspect, the clone policy 724 is associated with a backup policy 723 thatis described in more detail below. In another aspect, the backup policyis a part of the clone policy.

FIG. 7C shows an example of a process 748 to refresh a clone that ismanaged by SMS 132. The refresh process may be based on a schedule(721). When the refresh process is triggered, the cloning module 154queries (724) the stored cloning policy in data structure 176. Thedataset for the clone (726) and the latest clone object is queried aswell (728). The hosts that are involved with the cloning is identified(730). Thereafter, the clone is deleted (730). The process for deletingthe clone is described below with respect to FIG. 7F.

A backup for the dataset is then initiated (734) and the hosts that areidentified in the backup are identified (736). Thereafter, the backup isrun (738). The backup process has been described above with respect toFIGS. 5A-5C. The clone creation is initiated from the backup (740) andthe clone is created (742). Thereafter the clone refresh process iscompleted (744) and the metadata associated with the refreshed clone isupdated (746).

FIG. 7D shows an example of a high-level cloning process flow 750 forcloning a snapshot of a stored object, including a storage volume,according to one aspect. The process begins during operation 752 and inoperation 754, cloning module 154 of SMS 132 obtains the metadataregarding the backup that is being cloned. If the backup is from asecondary storage, then during operation 756, secondary backupinformation is obtained from SAL 151 that maintains all the storageresource information, as described above. A clone job is then createdduring operation 758. The job is uniquely identified and includes abackup identifier information.

The SMS 132 then requests clone creation to SMcore 130 during operation760. The SMcore 130 co-ordinates the clone creation with storage systeminterface and the appropriate application plugins that are involved. Itis noteworthy that at the storage system 120 level, the clone is createdby the storage services module 122. The clone is mounted duringoperation 762. The metadata for the clone (similar to the backupmetadata is registered during operation 764. The job status is updatedduring operation 766 and the process ends.

FIG. 7E-1/7E-3 shows an example of a clone workflow (or process flow)768 when a virtualization plugin is used in a host system, according toone aspect. The process flow shows interaction between SMS 132, SAL 151,SMcore 130, SAL 151A, VP (virtual plugin) 185A, storage system interface(shown as SnapDrive) 195 (or the file system plugin 195A) and anapplication plugin (for example, 193A-193C). The cloning createoperation 770 may be triggered for a live database, a clone of adatabase from a backup, as a scheduled operation or as part of clonelifecycle management. It is noteworthy that adaptive process is notlimited to any particular reason for triggering the clone createoperation.

SMS 132 obtains the backup metadata in operation 772. The backupmetadata is stored as part of database 176 or any other data structureas described above in detail.

If cloning is based on secondary storage, then during operation 773, SMS132 obtains the secondary storage information from SAL 151. SAL 151provides secondary storage information based on a primary storagebackup.

The job manager 182 then creates a clone job in operation 774. The jobobject stores the stored object identifier that is being cloned (forexample, a database), the host system that manages the stored object andother information. This allows one to track the clone job via a GUI, asdescribed below in detail.

During operation 776, the backup environment is verified by the SMcore130 to ensure that the clone operation can be completed. Duringoperation 778, host system information is obtained by the SMcore 130from the virtual plugin 185A, as well as from the storage systeminterface 195 (or SAL 151A). The host information includes hostidentifiers (host UUID) and host initiator information. Thereafter, thefile system is initialized during operation 782.

For a virtual environment, SMS 132 initiates a clone operation for thevirtual plugin 185A shown as operation 784. The host identifier providedby the storage system interface is passed to the virtual plugin 185A anda vdisk is attached to the host in operation 786. The metadata for theclone is then returned during operation 788.

During operation 790, the clone operation begins when the file system isbuilt during operation 792. In one aspect, the storage system interface(or SnapDrive) 195 builds the file system. The cloned metadata isreturned to SMcore 130 during operation 794. The application plugin thenreads the metadata and the application may be cloned during operation798/799 and the application may be recovered during 797. The clonedinstance is then returned during operation 795 to the SMcore 130.

FIG. 7F shows a clone delete process flow 749, according to one aspectof the present disclosure. The clone delete operation involves SMS 132,SMcore 130, the application plugins, storage system interface 195 andvirtual plugin 185A.

The process begins by cloning module 154 obtaining the clone metadataduring operation 751A and then requesting SMcore 130 to delete the clonein operation 751. The delete operation may be triggered by a user via aGUI, by a scheduled job or as part of clone life cycle management. Theclone metadata is similar to backup metadata described above in detail.The CloneId is selected from the metadata and provided to the SMcore130.

The SMcore 130 that interfaces with various application plugins issues adelete clone command in block 753. The application that manages theclone obtains the cloned application instance information from the clonemetadata during operation 755. The cloned application instance is thenshut down during 757. The application instance is first deleted andde-registered by the host operating system in 759. The SMcore layer 130then sends a delete clone request (761) to the storage system interface195. Using the metadata, the storage system interface 195 obtains thefile-system from the storage system 120. The file system is removed inoperation 765.

In a virtualized environment, the SMcore 130 requests SMS 130 to deletevdisks (767). The SMS 132 then issues a delete clone request (769) tothe virtual plugin 185A. The virtual plugin obtains the virtual diskinformation from the metadata (771), detaches the vdisk from the host(773) and delete or unmounts the vdisk so that it cannot be used (775).

The SMcore 130 obtains information regarding the cloned volumes and thecloned volumes are deleted (779). The SMcore 130 then issues apost-clone request (781) to the application plugin and the storagesystem interface 195. The clone metadata is then cleaned up and theprocess ends.

As described above, SMS 132, SAL 151, SAL 151A and other modules ofSMcore 130 are able to efficiently clone backups for differentapplications from a single management application. Regardless of how theapplications handle or manage data, because SMS 132 and SMcore 130communicate in the manner described above, the system is able to handlecloning and clone life cycle management for a plurality of applicationsand application objects.

GUI Examples:

In one aspect, GUI module 142 provides an integrated GUI as well asplugin specific GUIs. Backup, restore and cloning is managed from asingle pane, regardless of allocation and/or plugin type that is beingused at a host system. At a high level, the SMS 132 based GUI providesbasic administration and settings for the host systems and the storagesystem, plugin lifecycle management, an inventory view of pluginobjects, policy management across multiple plugins, dataset management,backup listing and management, restore operations, reporting and a jobmanager for displaying job details. It is noteworthy that the GUIprovided by SMS 132 may also include plugin GUI enhancements, forexample, plugin specific inventory views (for example, shares, databasesand others), policy enhancements/additions, dataset enhancements from aplugin's perspective as well as backup and restore operationrequirements.

FIGS. 8A-8T show examples of managing various storage service relatedoperations through a GUI that may be provided by GUI module 142 of SMS132, according to one aspect. The various screenshots are meant to be anexample of the various adaptive aspects disclosed herein and are notintended to limit the scope of the present disclosure.

FIG. 8A shows a high-level GUI dashboard that may be displayed on adisplay device of a user using SMS 132 to manage an environmentdescribed above with respect to FIGS. 1A and 1E. The left hand sidenavigation bar allows navigation to different views. It is noteworthythat the dashboard panel is configurable and plugins can add customdashboards. A user is also able to customize the dashboard view based ontheir preferences.

The dashboard 802 provides various selectable options or tabs, forexample, hosts 804, an inventory view option 806, datasets 808, policies810, jobs 812, reports 814, administration 816 and settings 818. In oneaspect, the GUI views are customized for users and their associatedroles that are maintained by the RBAC module 156.

The dashboard view shows different host systems that a user may bemanaging (819A), a consolidated view for the backups that may have beentaken (819B) and restore view status (819C). Dashboard 802 provides ahigh level view that can be modified by using the “modify” option. Thevarious views are based on database 176 maintained by SMS 132 forstoring information regarding various hosts, as described above indetail.

FIG. 8B shows an example of a host system view when a user selects thehost tab 804. The various displays are based on database 176 that aremaintained by the SM module 134. The host system view allows a user toadd any host using the add host option (804A) or remove a host using the“remove host” option (804B). The user can also view the detailsregarding each host, including the host name, operating system, if it isa part of a cluster and a port that is used to connect to the host.

Selectable options (or tabs) 804C, 804D, 804E, 804F and 804G provideother host related information. For example, 804C provides informationregarding the various storage disks for a host. From this option, disksmay be added or removed. CIFS share (storage objects) may be viewed andmodified using tab 804D. Storage volume information is obtained from tab804E which includes volume type, volume identifier, volume size andother details.

Initiator group (igroup) information may be obtained or modified usingtab 804F. Tab 804F allows a user to add or delete members of the igroupfrom the same GUI. iSCSI session information may be obtained using tab804G. iSCSI sessions are established to read or write data. iSCSI allowsa host to send SCSI commands over TCP/IP. The host system view alsoallows a user to manage plugins by using tab 804H. The user is able tosee all the plugins that are installed. If a plugin is needed then onecan add that using tab 804H.

FIG. 8C provides an inventory view to a user using a GUI to manage thesystem 100 of FIG. 1A. The inventory view provides various options,including adding a dataset using tab 806A, performing an on demandbackup using tab 806B, a restore operation using tab 806C, a refreshoperation for clone life cycle management using tab 806D, and deleting abackup using tab 806E. The inventory view provides name of storedobjects (for example, databases) under column 806F, the hosts that aremanaging the stored objects (column 806G), the owner of the database(column 806H), whether the database is a system database (8061), if thedatabase is selectable by the user (806J), if a dataset is associatedwith the database (806K), if a policy is associated with the database(806L), the last backup (806M) and the status of the last backup.

In one aspect, the inventory view may be customized to adapt to userroles. For example, if a user is not permitted to take a backup ondemand, then the user is not shown tab 806B.

The dataset view is shown in FIG. 8D. This screen allows the user tomanage datasets from a single screen. The user may add a new dataset(808A), modify it (808B), attach a policy (808C), detach a policy(808D), backup dataset (808E) and delete the dataset (808F). The list ofdatasets and their creation and modification dates are shown in columns808G, 808H, 8081 and 808J, respectively. In one aspect, the dataset viewmay also be customized to adapt to user roles.

FIG. 8E shows how a new dataset is added using tab 808A. The user isallowed to name the dataset, describe it, associate a policy andassociate a plugin. This format of the dataset allows SMS 132 to managebackups and other operations in a heterogeneous environment, regardlessof format or application type.

FIG. 8F provides a policies view, according to one aspect of the presentdisclosure. This screen allows an authorized user to manage policies fordifferent stored objects managed by different applications at differenthosts. A new policy may be added using tab 810A. A policy may be copiedusing tab 810B, modified using tab 810C and deleted using tab 810D.

Details of a policy may be obtained by clicking on the policy. Thedetails include a policy name (810E), schedule type (810F), description(810G), the date the policy is created and modified.

FIGS. 8G-8Q show various screen shots of a new policy wizard that allowsan authorized user to create a policy. FIG. 8G shows where a user hasselected tab 820A to name and describe a policy. FIG. 8H shows the useof tab 820B, where a schedule is associated with the policy. Theschedule includes recurrence settings and credentials of a user. FIG.8I, allows a user to specify a retention schedule using tab 820C. Theuser may specify a number of snapshots after which snapshots are deletedor a number of days.

FIG. 8J shows a screenshot that enables a user to replicate a backupfrom a first storage system (primary) to another storage system(secondary) using tab 820D. The replication process may use SnapMirrortechnology provided by NetApp Inc. or any other methodology. FIG. 8Kshows a screen shot that allows a user to add a script to the policyusing tab 820E. The backup scripts options are shown in FIG. 8K and areself-explanatory.

FIG. 8L enables a user to specify a backup type using tab 820F. Thebackup may be a full backup or a log backup. This option also allows auser to setup verification settings for a backup.

FIG. 8M enables a user to setup verification options using tab 820G. Theverification settings include the type of authentication that may beneeded (for example, SQL, Windows or others). The user may specific auser name and password for the verification.

FIG. 8N shows a screen shot for specifying availability group backupusing tab 8201. An availability group is used to provide reliability andredundancy, in case a primary system fails. The various options in FIG.8N are self-explanatory.

FIG. 8O shows a summary for the policy using tab 820J. The summaryprovides the policy name, description, backup type, triggers if any,expiration and retention.

FIG. 8P shows a job view (or pane), according to one aspect. The jobview is based on the job objects managed by the job manager 182. In oneaspect, all job information, regardless of application, database orstored object is maintained in the same format. This allows the user toreview the status of any job, regardless of the plugin, application orhost operating environment. A user may filter the jobs using the filtertab 812A. The filter tab allows the user to search for any job using ajob identifier or other parameters.

FIG. 8Q shows a reports pane (view), according to one aspect. The reportpane is presented by the reports module 172 of SM module 134 inconjunction with the GUI module. The reports module 172 uses database176 to generate various reports that are presented within the reportspane based on user roles. As an example, the reports pane presents thestatus of all backup jobs, including job status, start time, end timeand duration. The reports pane also provides restore status, if any forall backups. The reports pane also provides plugin specific report forany given plugin on a host system, as described below in detail.

In one aspect, a report is generated based on a request/responsestructure of FIG. 2A that is presented to or executed by the reportsmodule 172, for example, a “createbackupreport” request gathers data forbackups, “createrecoveryreport” request gathers information on restoreoperations, “createpluginreport” request gathers host plugin specificbackup data and “Createsummaryreport” request gathers job summary datafor a given day.

A createbackupreport request may specify a backupID, a time range,backup name, status, policy name, dataset name, resource name and a jobidentifier. The response is a list of backup reports based on the searchcriteria. The search criteria may be different for different plugins.

An example of a backup report includes an ID of a backup operation, jobID, protection group ID, policy ID, start and end time, protection groupname and policy name.

A user may also generate detailed reports, for example, a list ofobjects that were successfully backed up, list of objects that failedbackups, backup type, list of snapshots for an object that was backed upand other details.

A user is also able to obtain details about restore operations, forexample, restore operations that occurred before or after a certain timeand an ID of a restore job.

For a createpluginreport, the request may specify the name of a host, adataset, plugin type, day count and a terse i.e. restrict the amount ofdata returned. The response provides a total number of backups for theplugin, number of backups older than day counts, number of protectedobjects, number of unprotected objects, number of objects with failedbackups, number of objects with backups that are not protected tosecondary storage, total number of objects, the objects that are notbacked up, objects that are not part of a dataset and other information.

Referring back to FIG. 8Q, an authorized user may use the backup tab814A to obtain information on backups. The GUI presents a summary ofbackups indicating how many backups were successful and how many failed.The GUI also shows a backup trend for completion and failure. A user mayalso view the backup status (814E) as it pertains to a dataset (814C)and an associated policy (814D). SMS 132 is able to present thesereports because it manages applications/stored objects in aplatform/application independent format.

FIG. 8R shows a plugin report that a user can view using tab 814B,according to one aspect. The plugin report provides an overview (814F)for a particular plugin, for example, SQL. The user can see the totalbackups that are associated with the plugins, the number of failedbackups, unprotected databases and databases with unarchived backups.The overview enables a user to take preventive measures to protectdatabases. The protection status may be viewed as shown in 814G. Detailsof unprotected databases are shown in segment 814H and databases withfailed backups are shown in segment 814I.

The administrative pane is shown in FIG. 8S. This allows a user to addroles (816A), modify existing roles (816B), assign a role to a user(816C) or delete a role (816D). The user is able to modify roles bychanging permissions and attributes for different categories, includingdatasets, backup, restore, host systems and storage connections. Tab816E allows a user to assign resources from a single pane includingdatasets, host systems, storage connections and others. The resources inthis instance may be storage or networking resources.

The various plugin downloads may be viewed using tab 816F. The tabprovides a view to the user of different plugins that may have beendownloaded at different host systems.

FIG. 8T provides a settings pane, according to one aspect. The settingspane allows a user to manage VM credentials for all hosts. Tab 818A maybe used to establish new VM connection settings (for example, a storagevirtual machine), tab 818B is used to modify the settings and 818C maybe used to delete the settings. Details regarding a storage virtualmachine is also provided with the port connection and the protocol used.

In one aspect, a centralized server is provided to manage storageservices for different applications with different requirements andparameters. The centralized server is able to manage the variousoperations efficiently based on the uniform format used to manageinformation regarding stored objects, as described above in detail.

Storage System Node:

FIG. 9 is a block diagram of a system 900, according to one aspect.System 900 may be used by a stand-alone storage system 120 and/or astorage system node operating within a clustered storage system node forperforming various storage services described above. System 900communicates with SAL 151A and/or storage system interface 195 forproviding information regarding stored objects, backups, clones, restoreoperations and others as described above.

System 900 may include a plurality of processors 902A and 902B, a memory904, a network adapter 908, a cluster access adapter 912 (used for acluster environment), a storage adapter 916 and local storage 910interconnected by a system bus 906. The local storage 910 comprises oneor more storage devices, such as disks, utilized by the processors tolocally store configuration and other information.

The cluster access adapter 912 comprises a plurality of ports adapted tocouple system 900 to other nodes of a cluster (not shown). In theillustrative aspect, Ethernet may be used as the clustering protocol andinterconnect media, although it will be apparent to those skilled in theart that other types of protocols and interconnects may be utilizedwithin the cluster architecture described herein.

System 900 is illustratively embodied as a dual processor storage systemexecuting a storage operating system 124 that preferably implements ahigh-level module, such as a file system, to logically organizeinformation as a hierarchical structure of named directories, files andspecial types of files called virtual disks on storage devices 114.However, it will be apparent to those of ordinary skill in the art thatthe system 900 may alternatively comprise a single or more than twoprocessor systems.

The memory 904 illustratively comprises storage locations that areaddressable by the processors and adapters for storing programmableinstructions and data structures. The processor and adapters may, inturn, comprise processing elements and/or logic circuitry configured toexecute the programmable instructions and manipulate the datastructures. It will be apparent to those skilled in the art that otherprocessing and memory means, including various computer readable media,may be used for storing and executing program instructions pertaining tothe various aspects described herein.

The storage operating system 124, portions of which is typicallyresident in memory and executed by the processing elements, functionallyorganizes the system 900 by, inter alia, invoking storage operations insupport of the storage service provided by storage system 120. Anexample of operating system 124 is the DATA ONTAP® (Registered trademarkof NetApp, Inc. an operating system available from NetApp, Inc. thatimplements a Write Anywhere File Layout (WAFL® (Registered trademark ofNetApp, Inc.)) file system. However, it is expressly contemplated thatany appropriate storage operating system may be enhanced for use inaccordance with the inventive principles described herein. As such,where the term “ONTAP” is employed, it should be taken broadly to referto any storage operating system that is otherwise adaptable to theteachings of the various aspects disclosed herein.

The network adapter 908 comprises a plurality of ports adapted to couplethe system 900 to one or more clients (for example, hosts 102) overpoint-to-point links, wide area networks, virtual private networksimplemented over a public network (Internet) or a shared local areanetwork. The network adapter 908 thus may comprise the mechanical,electrical and signaling circuitry needed to connect storage system 120to the network. Illustratively, the computer network 109 may be embodiedas an Ethernet network or a FC network.

The storage adapter 916 cooperates with the storage operating system 124executing on the system 900 to access information requested by theclients and management application. The information may be stored on anytype of attached array of writable storage device media such as videotape, optical, DVD, magnetic tape, bubble memory, electronic randomaccess memory, flash memory devices, micro-electro mechanical and anyother similar media adapted to store information, including data andparity information.

The storage adapter 916 comprises a plurality of ports havinginput/output (I/O) interface circuitry that couples to the disks over anI/O interconnect arrangement, such as a conventional high-performance,Fibre Channel link topology. In another aspect, instead of using aseparate network and storage adapter, a converged adapter is used toprocess both network and storage traffic.

Storage Operating System:

FIG. 10 illustrates a generic example of storage operating system 124executed by storage system 120 and interfacing with storage systeminterface 195 (or SAL 151A), according to one aspect of the presentdisclosure. The storage operating system 124 maintains various storedobjects and data containers. Storage system interface 195 and/or SAL151A communicates with different layers of storage operating system 124for providing storage services in system 100, described above in detail.

As an example, storage operating system 124 may include several modules,or “layers”. These layers include a file system manager 1002 that keepstrack of a directory structure (hierarchy) of the data stored in storagedevices and manages read/write operations, i.e. executes read/writeoperations on disks in response to client requests.

Storage operating system 124 may also include a protocol layer 1004 andan associated network access layer 1008, to allow system 120 tocommunicate over a network with other systems. Protocol layer 1004 mayimplement one or more of various higher-level network protocols, such asNFS, CIFS, Hypertext Transfer Protocol (HTTP), TCP/IP and others, asdescribed below.

Network access layer 1008 may include one or more drivers, whichimplement one or more lower-level protocols to communicate over thenetwork, such as Ethernet. Interactions between clients (i.e. hostsystems/applications) and mass storage devices 114 are illustratedschematically as a path, which illustrates the flow of data throughstorage operating system 124.

The storage operating system 124 may also include a storage access layer1006 and an associated storage driver layer 1010 to communicate with astorage device. The storage access layer 1006 may implement ahigher-level disk storage protocol, such as RAID, while the storagedriver layer 1010 may implement a lower-level storage device accessprotocol, such as FC or SCSI.

It should be noted that the software “path” through the storageoperating system layers described above needed to perform data storageaccess for a client request may alternatively be implemented inhardware. That is, in an alternate aspect of the disclosure, the storageaccess request data path may be implemented as logic circuitry embodiedwithin a field programmable gate array (FPGA) or an ASIC. This type ofhardware implementation increases the performance of the file serviceprovided by storage system 120 in response to a file system requestissued by client 114.

As used herein, the term “storage operating system” generally refers tothe computer-executable code operable on a computer to perform a storagefunction that manages data access and may, in the case of system 120,implement data access semantics of a general purpose operating system.The storage operating system can also be implemented as a microkernel,an application program operating over a general-purpose operatingsystem, such as UNIX® or Windows XP®, or as a general-purpose operatingsystem with configurable functionality, which is configured for storageapplications as described herein.

In addition, it will be understood to those skilled in the art that thevarious aspects described herein may apply to any type ofspecial-purpose (e.g., file server, filer or storage serving appliance)or general-purpose computer, including a standalone computer or portionthereof, embodied as or including a storage system. Moreover, theteachings of this disclosure can be adapted to a variety of storagesystem architectures including, but not limited to, a network-attachedstorage environment, a storage area network and a disk assemblydirectly-attached to a client or host computer. The term “storagesystem” should therefore be taken broadly to include such arrangementsin addition to any subsystems configured to perform a storage functionand associated with other equipment or systems. It should be noted thatwhile this description is written in terms of a write anywhere filesystem, the teachings of the present aspects may be utilized with anysuitable file system, including a write in place file system.

Processing System:

FIG. 11 is a high-level block diagram showing an example of thearchitecture of a processing system, at a high level, in whichexecutable instructions as described above can be implemented. Theprocessing system 1100 can represent SMS 132, host system 102, VMM 106,virtual management console 103, and others. Note that certain standardand well-known components which are not germane to the various aspectsof this disclosure are not shown in FIG. 11.

The processing system 1100 includes one or more processors 1102 andmemory 1104, coupled to a bus system 1105. The bus system 1105 shown inFIG. 11 is an abstraction that represents any one or more separatephysical buses and/or point-to-point connections, connected byappropriate bridges, adapters and/or controllers. The bus system 1105,therefore, may include, for example, a system bus, a PeripheralComponent Interconnect (PCI) bus, a HyperTransport or industry standardarchitecture (ISA) bus, a small computer system interface (SCSI) bus, auniversal serial bus (USB), or an Institute of Electrical andElectronics Engineers (IEEE) standard 1394 bus (sometimes referred to as“Firewire”).

The processors 1102 are the central processing units (CPUs) of theprocessing system 1100 and, thus, control its overall operation. Incertain aspects, the processors 1102 accomplish this by executingprogrammable instructions stored in memory 1104. A processor 1102 maybe, or may include, one or more programmable general-purpose orspecial-purpose microprocessors, digital signal processors (DSPs),programmable controllers, application specific integrated circuits(ASICs), programmable logic devices (PLDs), or the like, or acombination of such devices.

Memory 1104 represents any form of random access memory (RAM), read-onlymemory (ROM), flash memory, or the like, or a combination of suchdevices. Memory 1104 includes the main memory of the processing system1100. Instructions 1106 which implement the various techniques/processflows introduced above may reside in and may be executed (by processors1102) from memory 1104. It is noteworthy that portions of instructions1106 may be executed by different processors out of different memorydevices.

Also connected to the processors 1102 through the bus system 1105 areone or more internal mass storage devices 1110, and a network adapter1112. Internal mass storage devices 1110 may be or may include anyconventional medium for storing large volumes of data in a non-volatilemanner, such as one or more magnetic or optical based disks. The networkadapter 1112 provides the processing system 1100 with the ability tocommunicate with remote devices (e.g., storage servers) over a networkand may be, for example, an Ethernet adapter, a FC adapter, or the like.The processing system 1100 also includes one or more input/output (I/O)devices 1108 coupled to the bus system 1105. The I/O devices 1108 mayinclude, for example, a display device, a keyboard, a mouse, etc.

Cloud Computing:

The system and techniques described above are applicable and especiallyuseful in the cloud computing environment where storage is presented andshared across different platforms. Cloud computing means computingcapability that provides an abstraction between the computing resourceand its underlying technical architecture (e.g., servers, storage,networks), enabling convenient, on-demand network access to a sharedpool of configurable computing resources that can be rapidly provisionedand released with minimal management effort or service providerinteraction. The term “cloud” is intended to refer to a network, forexample, the Internet and cloud computing allows shared resources, forexample, software and information to be available, on-demand, like apublic utility.

Typical cloud computing providers deliver common business applicationsonline which are accessed from another web service or software like aweb browser, while the software and data are stored remotely on servers.The cloud computing architecture uses a layered approach for providingapplication services. A first layer is an application layer that isexecuted at client computers. In this example, the application allows aclient to access storage via a cloud.

After the application layer, is a cloud platform and cloudinfrastructure, followed by a “server” layer that includes hardware andcomputer software designed for cloud specific services. The storagesystems described above can be a part of the server layer for providingstorage services. Details regarding these layers are not germane to theinventive aspects.

Thus, a method and apparatus for managing storage services has beendescribed. Note that references throughout this specification to “oneaspect” or “an aspect” mean that a particular feature, structure orcharacteristic described in connection with the aspect is included in atleast one aspect of the present disclosure. Therefore, it is emphasizedand should be appreciated that two or more references to “an aspect” or“one aspect” or “an alternative aspect” in various portions of thisspecification are not necessarily all referring to the same aspect.Furthermore, the particular features, structures or characteristicsbeing referred to may be combined as suitable in one or more aspects ofthe present disclosure, as will be recognized by those of ordinary skillin the art.

While the present disclosure is described above with respect to what iscurrently considered its preferred aspects, it is to be understood thatthe disclosure is not limited to that described above. To the contrary,the disclosure is intended to cover various modifications and equivalentarrangements within the spirit and scope of the appended claims.

What is claimed is:
 1. A method, comprising: utilizing by a managementapplication of a device a same format for integrated objects associatedwith different applications that use different application objects tostore data at a storage system, each integrated object indicating astorage protocol, usage of a virtual resource, usage of a physicalresource, and a storage share type used for storing data for one of theapplications; wherein the management application of the devicecommunicates with a management layer executed by a computing deviceusing a same request and response format regardless of an applicationtype, the management layer communicating with a plurality of pluginsexecuted by the computing device, each plugin corresponding to anapplication and used for providing storage services for the data storedby the storage system; retrieving a first integrated object by themanagement layer for a storage object of a first application, inresponse to a backup operation for the storage object initiated by themanagement application of the device using a backup policy having afirst portion that is same for different storage object types and asecond portion customized for the first application; wherein themanagement application notifies the management layer of the backupoperation and the management layer then interfaces with a first pluginassociated with the first application and a second plugin associatedwith the storage system to retrieve the first integrated object from thestorage system; executing the backup operation by the storage system inresponse to a request from the second plugin based on the firstintegrated object, after the first plugin corresponding to the firstapplication is quiesced by the management layer; obtaining backupmetadata associated with the backup operation by the management layerfrom the device for a restore operation initiated by the managementapplication of the device, wherein the management application of thedevice stores backup metadata for different applications in a sameformat identifying backup policy, plug-in type, and an attribute forcustomizing backup metadata for the different applications; and usingthe backup metadata by the management layer to interface with the firstplugin and the second plugin to execute the restore operation.
 2. Themethod of claim 1, wherein the different storage protocols include aCommon Internet File System (CIFS), a Network File System (NFS) and avirtual disk system.
 3. The method of claim 1, wherein the secondportion of the backup policy identifies a verification policy fordefining backup verification of the storage object.
 4. The method ofclaim 1, wherein the backup metadata includes an operations fieldidentifying operations for executing the backup operation.
 5. The methodof claim 1, wherein the device is a management device that initiates adiscovery operation and the management layer executed by the computingdevice obtains computing device information, application information andstorage resource information for storage resources used by eachapplication and provides the obtained information to the device in asame format, regardless of information type.
 6. The method of claim 1,wherein for different storage objects, a same format dataset is used tomanage backup, clone and restore operations for the differentapplications.
 7. The method of claim 1, wherein the device provides agraphical user interface for creating a dataset for the storage objectand associating a policy for managing backup, clone and restoreoperations.
 8. A non-transitory, machine readable storage medium havingstored thereon instructions for performing a method, comprising machineexecutable code which when executed by at least one machine, causes themachine to: utilize by a management application of a device a sameformat for integrated objects associated with different applicationsthat use different application objects to store data at a storagesystem, each integrated object indicating a storage protocol, usage of avirtual resource, usage of a physical resource, and a storage share typeused for storing data for one of the applications; wherein themanagement application of the device communicates with a managementlayer executed by a computing device using a same request and responseformat regardless of an application type, the management layercommunicating with a plurality of plugins executed by the computingdevice, each plugin corresponding to an application and used forproviding storage services for the data stored by the storage system;retrieve a first integrated object by the management layer for a storageobject of a first application, in response to a backup operation for thestorage object initiated by the management application of the deviceusing a backup policy having a first portion that is same for differentstorage object types and a second portion customized for the firstapplication; wherein the management application notifies the managementlayer of the backup operation and the management layer then interfaceswith a first plugin associated with the first application and a secondplugin associated with the storage system to retrieve the firstintegrated object from the storage system; execute the backup operationby the storage system in response to a request from the second pluginbased on the first integrated object, after the first plugincorresponding to the first application is quiesced by the managementlayer; obtain backup metadata associated with the backup operation bythe management layer from the device for a restore operation initiatedby the management application of the device, wherein the managementapplication of the device stores backup metadata for differentapplications in a same format identifying backup policy, plug-in type,and an attribute for customizing backup metadata for the differentapplications; and use the backup metadata by the management layer tointerface with the first plugin and the second plugin to execute therestore operation.
 9. The storage medium of claim 8, wherein thedifferent storage protocols include a Common Internet File System(CIFS), a Network File System (NFS) and a virtual disk system.
 10. Thestorage medium of claim 8, wherein the second portion of the backuppolicy identifies a verification policy for defining backup verificationof the storage object.
 11. The storage medium of claim 8, wherein thebackup metadata includes an operations field identifying operations forexecuting the backup operation.
 12. The storage medium of claim 8,wherein the device is a management device that initiates a discoveryoperation and the management layer executed by the computing deviceobtains computing device information, application information andstorage resource information for storage resources used by eachapplication and provides the obtained information to the device in asame format, regardless of information type.
 13. The storage medium ofclaim 8, wherein for different storage objects, a same format dataset isused to manage backup, clone and restore operations for the differentapplications.
 14. The storage medium of claim 13, wherein the deviceprovides a graphical user interface for creating a dataset for thestorage object and associating a policy for managing backup, clone andrestore operations.
 15. A system, comprising: a memory containingmachine readable medium comprising machine executable code having storedthereon instructions; and a processor coupled to the memory, theprocessor configured to execute the machine executable code to: utilizeby a management application of a management device a same format forintegrated objects associated with different applications that usedifferent application objects to store data at a storage system, eachintegrated object indicating a storage protocol, usage of a virtualresource, usage of a physical resource, and a storage share type usedfor storing data for one of the applications; wherein the managementapplication communicates with a management layer executed by a computingdevice using a same request and response format regardless of anapplication type, the management layer communicating with a plurality ofplugins executed by the computing device, each plugin corresponding toan application and used for providing storage services for the datastored by the storage system; retrieve a first integrated object by themanagement layer for a storage object of a first application, inresponse to a backup operation for the storage object initiated by themanagement application using a backup policy having a first portion thatis same for different storage object types and a second portioncustomized for the first application; wherein the management applicationnotifies the management layer of the backup operation and the managementlayer then interfaces with a first plugin associated with the firstapplication and a second plugin associated with the storage system toretrieve the first integrated object from the storage system; executethe backup operation by the storage system in response to a request fromthe second plugin based on the first integrated object, after the firstplugin corresponding to the first application is quiesced by themanagement layer; obtain backup metadata associated with the backupoperation by the management layer from the management device for arestore operation initiated by the management application of themanagement device, wherein the management application of the devicestores backup metadata for different applications in a same formatidentifying backup policy, plug-in type, and an attribute forcustomizing backup metadata for the different applications; and use thebackup metadata by the management layer to interface with the firstplugin and the second plugin to execute the restore operation.
 16. Thesystem of claim 15, wherein the different storage protocols include aCommon Internet File System (CIFS), a Network File System (NFS) and avirtual disk system.
 17. The system of claim 15, wherein the secondportion of the backup policy identifies a verification policy fordefining backup verification of the storage object.
 18. The system ofclaim 15, wherein the backup metadata includes an operations fieldidentifying operations for executing the backup operation.
 19. Thesystem of claim 15, wherein for different storage objects, a same formatdataset is used to manage backup, clone and restore operations for thedifferent applications.
 20. The system of claim 19, wherein themanagement device provides a graphical user interface for creating adataset for the storage object and associating a policy for managingbackup, clone and restore operations.